Zhiqiang::He

Message 2 in thread
寄件者：Wietse Venema (wietse@porcupine.org)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-12 17:22:01 PST

Why does mkmap_cdb_open() lock the .cdb file?

Wietse

Michael Tokarev:
> After recent comparisions and mention of CDB map type, I received
> several emails from several people about my tinycdb package. I've
> made cosmetic changes in packaging scripts and uploaded binaries for
> RedHat-like (rpm) and Debian Linux systems. It is available at
> ftp://ftp.corpit.ru/pub/tinycdb/ :
>
> 21985 tinycdb-0.72.tar.gz source tarball incl. build scripts
> 26572 tinycdb-0.72-1.i386.rpm RedHat i386 rpm (rpm -tb)
> 24064 tinycdb_0.72_i386.deb Debian i386 .deb (native build)
>
> This includes both the -lcdb library and simple cdb command-line
> utility to work with .cdb files.
>
> CDB map for postfix is available at ftp://ftp.corpit.ru/pub/postfix/,
> see README.CDB in this directory.
>
> /mjt
>
> -
> To unsubscribe, send mail to majordomo@postfix.org with content
> (not subject): unsubscribe postfix-users
>
>

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Message 3 in thread
寄件者：Michael Tokarev (mjt@tls.msk.ru)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-12 18:32:02 PST

Wietse Venema wrote:
> Why does mkmap_cdb_open() lock the .cdb file?

Hmm, this is an interesting question - now, after more than a
year since that discussions, I actually see what did you mean
when asked similar question.

There are two things to protect by a lock: creating a map file
updating of a map - this last one in case of cdb consists of
two operations, namely creating and renaming. I.e., new (temp)
file may be protected during it's creation/writing so no two
processes will write to it at once, or the whole operation
may be protected, so no two proceses will (create+rename) at
once.

Or, in the other words, lock on .cdb file should mean "wait,
I'm working on this map", while a lock on .tmp file should
mean "wait, I'm working with this file".

There is a minor and somewhat unclean difference, but it
exists.

I don't know which is better - in fact, there is no real
difference in a result, it seems. Why? Because if the
only program who'll update a map will be postfix, there is
really no difference once all postfix's components are
consistent. But if there will be other programs, things
will be more "interesting": there is no "standard" choice
for a name of a temp file, one program may use .cdb.tmp
(as my mkmap_cdb), another may use .tmp, yet another may
use random-unique-name etc. No other tool I know of uses
any locking at all when creating cdb (there are only 2:
DJB's cdbmake and tinycdb's cdb -c).

Well, if some other program will actually do soome locking,
it's a good idea to choose the same locking scheme, and the
right one (from the CDB's ideology) seems to be to lock .tmp
file ("I'm working on this file").

But implementation is more difficult in this case (when locking
.tmp) - it's more tricky to avoid a race here and not to start
updating .cdb file instead of .tmp -- if memory serves me right;
it was long ago when I thought about this. This was the only
reason why I choosed to lock .cdb instead of .tmp when first
wrote the map. Anyway, it shouldn't be impossible ;)

BTW, the whole cdb thing, just like maildir format, seems to
be designed to avoid any and all locking altogether. For this,
simplest solution is to choose a random (unique) name of temp
file and forget about concurrency issues. This solution has
one implication (already noted by you) - it's possible to have
leftover files after crashes in this case.

/mjt

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Message 4 in thread
寄件者：Matthias Andree (ma@dt.e-technik.uni-dortmund.de)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-14 09:00:06 PST

Michael Tokarev writes:

> There are two things to protect by a lock: creating a map file
> updating of a map - this last one in case of cdb consists of
> two operations, namely creating and renaming. I.e., new (temp)
> file may be protected during it's creation/writing so no two
> processes will write to it at once, or the whole operation
> may be protected, so no two proceses will (create+rename) at
> once. ...
> BTW, the whole cdb thing, just like maildir format, seems to
> be designed to avoid any and all locking altogether.

Not really, but that's why you pass DJB's applications the name of a
temp file. Whether you let the user choose one or use mkstemp, is a
matter of convenience and portability.

> For this, simplest solution is to choose a random (unique) name of
> temp file and forget about concurrency issues. This solution has one
> implication (already noted by you) - it's possible to have leftover
> files after crashes in this case.

Do you think it would be possible to use mkstemp or something for the
new map file and rename(2) that into place after it is complete, and
dropping all locking code?

I'd not really worry about "two processes rename to the same .cdb file"
at all: let the last one to rename win. Usually, these .cdb files are
under administrator control and will actually be built from some input
by a Makefile and make or something.

You could just document the template you passed to mkstemp() and tell
users they can safely delete these files. Should this happen while your
makecdb is running at the same time, rename() will fail without touching
the destination file.

No need to be paranoid here.

--
Matthias Andree
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 5 in thread
寄件者：Victor.Duchovni@morganstanley.com (Victor.Duchovni@morganstanley.com)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-14 10:12:01 PST

On Mon, 14 Oct 2002, Matthias Andree wrote:

> I'd not really worry about "two processes rename to the same .cdb file"
> at all: let the last one to rename win. Usually, these .cdb files are
> under administrator control and will actually be built from some input
> by a Makefile and make or something.
>

I don't think this is right. When I run two concurrent copies of "postmap
-i" I expect both sets of key/value pairs to be added to the map.

Does the cdb "mkmap" code support incremental inserts or only full map
rebuilds?

--
Viktor.

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 6 in thread
寄件者：Michael Tokarev (mjt@tls.msk.ru)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-14 12:28:07 PST

Victor.Duchovni@morganstanley.com wrote:
[]
> I don't think this is right. When I run two concurrent copies of "postmap
> -i" I expect both sets of key/value pairs to be added to the map.
>
> Does the cdb "mkmap" code support incremental inserts or only full map
> rebuilds?

No - this is a principal limitation of a cdb, it's a Constant database.
I.e. any and all manipulation with content requires complete rebuild.
Well, not strictly this - updates are possible by rebuilding the whole
hash table (inplace) or by recreating a file using old file as a source,
but that's not worth the effort to program the whole thing IMHO. For
this very reason, cdb isn't sutable for pop-before-smtp and the like
dynamic maps.

Mkmap code in my cdb patch does not support incremental updates, again,
for the same reason (it will be complete rebuild anyway).

/mjt

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 7 in thread
寄件者：Bennett Todd (bet@rahul.net)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-15 12:24:05 PST

--n8g4imXOkfNTN/H1
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

2002-10-14-15:24:50 Michael Tokarev:
> No - this is a principal limitation of a cdb, it's a Constant
> database. I.e. any and all manipulation with content requires
> complete rebuild. [...] For this very reason, cdb isn't sutable
> for pop-before-smtp and the like dynamic maps.

Not to dispute any of the other stuff you said, but I'm really not
sure about this bit.

I'd have to do some experimenting, but I really suspect
pop-before-smtp would work very well indeed with cdb.

It might be worth taking the trouble to have a version of cdb -c
inlined into the daemon, perhaps make a dynamically loadable perl
module if I were doing this on my pop-before-smtp daemon. That'd
save the need for fork-n-exec for each db update; if you ain't on
Linux, fork-n-exec is overpriced:-).

But with an in-memory cdb create, the update would be
open-tmp-file/write/close/rename, and for reasonable size maps ---
many folks don't have that many concurrent poppers from outside
mynetworks --- I'm sure this would be cheaper than
open-db/lock/read/write/flush/unlock/close. In fact, I wouldn't be
surprised if the break-even point for an optimized cdb-writer
weren't actually pretty darned large. cdb is awfully quick.

Hmm. Now I'm gonna have to find time to play with CDB_File.

-Bennett

--n8g4imXOkfNTN/H1
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9rF82HZWg9mCTffwRAuGhAJ9×4iqOrAmlmsXdy0QpPHFFFFaDCgCgjumd
Dpbm/WiCJre7+sXb9TCRJ/g=
=AMc7
-----END PGP SIGNATURE-----

n8g4imXOkfNTN/H1
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 8 in thread
寄件者：Michael Tokarev (mjt@tls.msk.ru)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-15 12:46:02 PST

Bennett Todd wrote:
> 2002-10-14-15:24:50 Michael Tokarev:
>
>>No - this is a principal limitation of a cdb, it's a Constant
>>database. I.e. any and all manipulation with content requires
>>complete rebuild. [...] For this very reason, cdb isn't sutable
>>for pop-before-smtp and the like dynamic maps.
>
>
> Not to dispute any of the other stuff you said, but I'm really not
> sure about this bit.
>
> I'd have to do some experimenting, but I really suspect
> pop-before-smtp would work very well indeed with cdb.
>
> It might be worth taking the trouble to have a version of cdb -c
> inlined into the daemon, perhaps make a dynamically loadable perl
> module if I were doing this on my pop-before-smtp daemon. That'd
> save the need for fork-n-exec for each db update; if you ain't on
> Linux, fork-n-exec is overpriced:-).
>
> But with an in-memory cdb create, the update would be
> open-tmp-file/write/close/rename, and for reasonable size maps ---
> many folks don't have that many concurrent poppers from outside
> mynetworks --- I'm sure this would be cheaper than
> open-db/lock/read/write/flush/unlock/close. In fact, I wouldn't be
> surprised if the break-even point for an optimized cdb-writer
> weren't actually pretty darned large. cdb is awfully quick.

One word: scalability. With a dynamically updateable map, a
time needed to insert one record does not depend on the number
of records already present in a map (well, almost). Cdb
requires complete rewrite. It will work for small sites.
Also, create/rename requires filesystem metadata updates,
while read/write does not - thus, depending on a filesystem,
situation with cdb will be worse. But it has one advantage:
it will work well over NFS, so it will be possible to use
PBS when pop daemon isn't on the same machine as smtp.

In fact, the best way to implement PBS is to keep all records
in memory (8 bytes for each (IP and time) isn't that much)
and use tcp map in postfix (and this too will work good
on a large site). Crashes/reboots aren't that important
here, it's almost ok to lose data since it isn't keept
for a long time anyway (just "rePOP" will be required).
With such a method, far more efficient search algorithm
may be implemented (i.e. sorted array of IP addresses,
maybe hashed).

> Hmm. Now I'm gonna have to find time to play with CDB_File.

I do NOT recommend this, at least w/o fixing things first.
Cdbmake code has at least one (huge) memory leak and thus
isn't sutable for a long-living daemon. Also, cdbmake code
requires much more memory than tinycdb, and is less
efficient (this all is true for cdb-0.75, I not looked
if new version was released).

What I want to is to find a time to write perl and especially
nss module for cdb - this is on a todo list for more than a
year now... ;)

/mjt

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
寄件者：Bennett Todd (bet@rahul.net)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-16 09:40:03 PST

--kXdP64Ggrk/fb43R
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

2002-10-15-15:43:50 Michael Tokarev:
> Bennett Todd wrote:
> >I'd have to do some experimenting, but I really suspect
> >pop-before-smtp would work very well indeed with cdb.
>=20
> One word: scalability. With a dynamically updateable map, a
> time needed to insert one record does not depend on the number
> of records already present in a map (well, almost). Cdb
> requires complete rewrite. It will work for small sites.

Define "small".

I seriously believe that a large majority of pop-before-smtp users
rarely have more than a few records in their database; and that
sites with as many as a hundred such records much of the time are
liable to be a teensy minority, if not a non-existent limit case.

Where's the crossover point? I.e. where does the cost of the CDB
generation grow to exceed the cost of doing the DB manipulation?

This undoubtedly varies from platform to platform. I'll try and
produce an answer for at least one platform:-).

> Also, create/rename requires filesystem metadata updates,
> while read/write does not - thus, depending on a filesystem,
> situation with cdb will be worse.

I'll try it with ext3, and with tmpfs. But while read/write may not
require metadata updates, it does require locking. I don't know
what sort of difference that will make, if any.

> In fact, the best way to implement PBS is to keep all records
> in memory (8 bytes for each (IP and time) isn't that much)
> and use tcp map in postfix (and this too will work good
> on a large site).

Sounds like we're converging on DRAC:-).

> >Hmm. Now I'm gonna have to find time to play with CDB_File.
>=20
> I do NOT recommend this, at least w/o fixing things first.
> Cdbmake code has at least one (huge) memory leak and thus
> isn't sutable for a long-living daemon.
观看文件全部内容 （仍有 22 行）

Post a follow-up to this message

Message 10 in thread
寄件者：Matthias Andree (ma@dt.e-technik.uni-dortmund.de)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 06:25:57 PST

Bennett Todd writes:

> I'd have to do some experimenting, but I really suspect
> pop-before-smtp would work very well indeed with cdb.

pop-before-smtp is inherently insecure, no matter HOW you implement
it. The problem is that you give relay permission to a host that may
well be "replaced" by some other (1st hosts disconnects, 2nd is assigned
the previous IP from 1st host from the dialup pool), without your SMTP
server noticing the disconnect. SMTP AUTH does not suffer from these
problems and does not need all these expire cronjobs and tossing data
from POP server to SMTP server and the like, worrying about how you
update a cdb... A cdb does not cut incremental updates. It would cope
well with a passwd like file though.

--
Matthias Andree
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users

寄件者：Bennett Todd (bet@rahul.net)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 07:44:59 PST

--gTtJ75FAzB1T2CN6
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline

2002-10-21-08:59:40 Matthias Andree:
> pop-before-smtp is inherently insecure, no matter HOW you implement
> it.

Lightly so, yes.

> The problem is that you give relay permission to a host that may
> well be "replaced" by some other (1st hosts disconnects, 2nd is assigned
> the previous IP from 1st host from the dialup pool), without your SMTP
> server noticing the disconnect.

That's why there's a timeout. A default of 1/2 hour seems to be
fine. Yes, there's a window during which someone else who got the IP
addr of a recently-pop-authed machine could relay. No, it doesn't
appear to be a problem in practice; I've never heard of such a setup
actually being exploited through this timing window.

> SMTP AUTH does not suffer from these problems [...]

If SMTP AUTH didn't require SASL, or if there were an implementation
of SASL besides Cyrus, I might be more interested. Is it possible to
implement enough SASL for SMTP AUTH without bringing in mountains of
excruciatingly painful goo, like e.g. GSSAPI with it's ASN.1 and so
forth?

> [...] and does not need all these expire cronjobs and tossing data
> from POP server to SMTP server and the like, [...]

I don't have any expire cronjobs, I just have a single persistent
daemon that manages the data file. Works great for a single server
that's both the pop/imap mailbox server and the smtp relay server.
For server farms, something more like DRAC or whoson would probably
be in order.

> worrying about how you update a cdb... A cdb does not cut
> incremental updates.

Sure it does, it does so beautifully. cdb file rebuild is so fast
that doing a full rebuild for each update is practical in many
settings --- as DNS hosting sites have discovered when using
tinydns.

-Bennett

--gTtJ75FAzB1T2CN6
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9tA7UHZWg9mCTffwRAhs7AKDEw1uALoCxjHAhlSJA0xic+2hfTACgm6a2
0BJdYgRUKo90nwQlSFdkK24=
=XNg1
-----END PGP SIGNATURE-----

gTtJ75FAzB1T2CN6
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 12 in thread
寄件者："Schmehl, Paul L" (pauls@utdallas.edu)
主旨：RE: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 08:30:04 PST

> -----Original Message-----
> From: Bennett Todd [mailto:bet@rahul.net]
> Sent: Monday, October 21, 2002 9:28 AM
> To: Matthias Andree
> Cc: postfix-users@postfix.org
> Subject: Re: [ot] tinyCDB is available in packaged form
>
>
> 2002-10-21-08:59:40 Matthias Andree:
> > pop-before-smtp is inherently insecure, no matter HOW you implement
> > it.
>
> Lightly so, yes.
>
> > The problem is that you give relay permission to a host that may well
> > be "replaced" by some other (1st hosts disconnects, 2nd is assigned
> > the previous IP from 1st host from the dialup pool), without your SMTP
> > server noticing the disconnect.
>
> That's why there's a timeout. A default of 1/2 hour seems to
> be fine. Yes, there's a window during which someone else who
> got the IP addr of a recently-pop-authed machine could relay.
> No, it doesn't appear to be a problem in practice; I've never
> heard of such a setup actually being exploited through this
> timing window.

There's quite an assumption behind this claim of insecurity. The
assumption is:

1) User one connects using pop before SMTP (or DRAC), reads mail and
disconnects before the timeout.
2) User two knows he a) has user one's IP address and knows b) the
IP/ hostname of the mail server and knows c) the server is open for
relay and intends d) to send spam through the server.

This assumption requires some incredible leaps in logic. Is it
possible? Of course. Just as breaking a 4096 AES key is possible. Is
it likely? Not in this lifetime. Even if I were a spammer, and even if
I knew that a particular server used pop before SMTP or DRAC and even
if I knew someone who used that server was online and using the server
and even if I knew precisely when they disconnected, I would still
have to obtain the exact same IP address to exploit the server.

I have real problems thinking of a scenario where this is even remotely
possible.

Paul Schmehl (pauls@utdallas.edu)
TCS Department Coordinator
The University of Texas at Dallas
AVIEN Founding Member
http://www.utdallas.edu/~pauls/
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 13 in thread
寄件者：Simon White (simon@mtds.com)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 08:32:05 PST

21-Oct-02 at 10:19, Schmehl, Paul L (pauls@utdallas.edu) wrote :

> 1) User one connects using pop before SMTP (or DRAC), reads mail and
> disconnects before the timeout.
> 2) User two knows he a) has user one's IP address and knows b) the
> IP/ hostname of the mail server and knows c) the server is open for
> relay and intends d) to send spam through the server.
>
> This assumption requires some incredible leaps in logic. Is it
> possible? Of course. Just as breaking a 4096 AES key is possible. Is
> it likely? Not in this lifetime. Even if I were a spammer, and even if
> I knew that a particular server used pop before SMTP or DRAC and even
> if I knew someone who used that server was online and using the server
> and even if I knew precisely when they disconnected, I would still
> have to obtain the exact same IP address to exploit the server.
>
> I have real problems thinking of a scenario where this is even remotely
> possible.

Unless you have ISDN users with a fixed IP address : but then you'd have
to sniff their dialup username and password into the bargain.

--
[Simon White. vim/mutt. simon@mtds.com. GIMPS:41.94% see www.mersenne.org]
/"\ ASCII Ribbon Campaign
\ / Respect for open standards
X No HTML/RTF in email
/ \ No M$ Word docs in email
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 14 in thread
寄件者：Michael Tokarev (mjt@tls.msk.ru)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 09:44:04 PST

Matthias Andree wrote:
[]
> pop-before-smtp is inherently insecure, no matter HOW you implement
> it. The problem is that you give relay permission to a host that may
> well be "replaced" by some other (1st hosts disconnects, 2nd is assigned
> the previous IP from 1st host from the dialup pool), without your SMTP
> server noticing the disconnect.

Mattias, this is not an issue at all with current world. Any given
(dialup) host has several minutes (hours? maybe days?) to find out
which mailservers was used by a host that has this IP previously,
from about 2^32 other hosts on the 'net. This is not possible in
that timeframe, and there are way far simpler ways exists to find
other open relays. Granted, if such PBS records will be permanent,
at some time a spammer eventually will find a relay, but that's a)
not the case (records aren't permanent), and b) chances are very
low anyway. Having a timeout of about an hour makes zero chances
for spammers to reuse the "relay". Again, in some cases, PBS is
somewhat insecure - think of common well-known public mail
systems like e.g. mail.com (a dialup spammer may try to send out
spam via mail.com just after dialing in to see if some previous
user used mail.com from this IP address recently), but again,
such cases are rare, and public mail systems should use other
protection methods as well (i.e. limiting number of recipients
in a time period, have smaller timeout for PBS etc).

/mjt

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 15 in thread
寄件者："Devin L. Ganger" (devin@thecabal.org)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 13:20:05 PST

On Mon, Oct 21, 2002 at 10:19:04AM -0500, Schmehl, Paul L wrote:

> There's quite an assumption behind this claim of insecurity. The
> assumption is:
>
> 1) User one connects using pop before SMTP (or DRAC), reads mail and
> disconnects before the timeout.

True.

> 2) User two knows he a) has user one's IP address and knows b) the
> IP/ hostname of the mail server and knows c) the server is open for
> relay and intends d) to send spam through the server.

False. All user two has to do is attempt to relay. I wouldn't be at
all surprised to find that as more and more ISPs start protecting
their servers from relay, we're going to see more and more examples
of this from whackamole accounts.

--
Devin L. Ganger
Co-Admin, The cabalSASL Project ( http://sasl.thecabal.org/ )
A man, a miss, a car -- a curve,
He kissed the miss and missed the curve -- Burma Shave (1948)
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 16 in thread
寄件者："Schmehl, Paul L" (pauls@utdallas.edu)
主旨：RE: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 14:16:04 PST

Why do you say this? If I attempt to relay through a server that uses
pbfs, it will fail - unless I am using the correct IP. So I not only
have to have the mail server's address, but I also must "own" the IP
address that's authorized to relay - or do you know something no one
else does about pbfs?

You can try relaying all day long, but unless you have the IP, you have
nothing. If you have the IP, then you must also be trying to relay
through that mail server. No other server will do. You make it sound
as though it's trivial to do. I don't see that at all.

Perhaps you could elaborate further?

Paul Schmehl (pauls@utdallas.edu)
TCS Department Coordinator
The University of Texas at Dallas
AVIEN Founding Member
http://www.utdallas.edu/~pauls/

> -----Original Message-----
> From: Devin L. Ganger [mailto:devin@thecabal.org]
> Sent: Monday, October 21, 2002 3:11 PM
> To: postfix-users@postfix.org
> Subject: Re: [ot] tinyCDB is available in packaged form
>
>
> On Mon, Oct 21, 2002 at 10:19:04AM -0500, Schmehl, Paul L wrote:
>
> > There's quite an assumption behind this claim of insecurity. The
> > assumption is:
> >
> > 1) User one connects using pop before SMTP (or DRAC), reads mail and
> > disconnects before the timeout.
>
> True.
>
> > 2) User two knows he a) has user one's IP address and knows b) the
> > IP/ hostname of the mail server and knows c) the server is open for
> > relay and intends d) to send spam through the server.
>
> False. All user two has to do is attempt to relay. I
> wouldn't be at all surprised to find that as more and more
> ISPs start protecting their servers from relay, we're going
> to see more and more examples of this from whackamole accounts.
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 17 in thread
寄件者：Simon White (simon@mtds.com)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 15:46:02 PST

21-Oct-02 at 13:10, Devin L. Ganger (devin@thecabal.org) wrote :
> On Mon, Oct 21, 2002 at 10:19:04AM -0500, Schmehl, Paul L wrote:
>
> > There's quite an assumption behind this claim of insecurity. The
> > assumption is:
> >
> > 1) User one connects using pop before SMTP (or DRAC), reads mail and
> > disconnects before the timeout.
>
> True.
>
> > 2) User two knows he a) has user one's IP address and knows b) the
> > IP/ hostname of the mail server and knows c) the server is open for
> > relay and intends d) to send spam through the server.
>
> False. All user two has to do is attempt to relay. I wouldn't be at
> all surprised to find that as more and more ISPs start protecting
> their servers from relay, we're going to see more and more examples
> of this from whackamole accounts.
>

Indeed, if a spammer already has an inkling of a dialup subnet range
with the same domain name, it is quite likely that he will already have
been able to send spam through an open proxy which is purely a poor SME
connection sharing thing with some Exchange administrator who wouldn't
even pass an MCSE... although Microsoft have made some progress in this
direction there's a cost to upgrade unlike Open Source which offers free
upgrades in return for a bit of clue and your time.

Then he can regularly attempt to find a connection with an IP in the
POP-before-SMTP database but this is of little use to him: he might more
easily forge something and get past a poor server rather than look for
30 minute windows in which to inject as much spam as possible. With a
solid mailserver setup as any self-respecting mail admin should have,
this is less /and more!/ of a problem.

Sadly, there are more people in this world interested in finding mugs,
and they do find them, so spam is here to stay. As much as you'd have
many avenues to explore before feeling obliged to hack enough just to
get a maximum of one half hour of time, an admin proud of his Postfix
server which could (and has done, on my site, once...) deliver something
like 15,000 messages in a short period of time, on a reasonably fat pipe
to the Internet, might think that he is indeed prime spam territory. Of
course this is all relative because here fat is 2mbps!

Anyway anecdotes aside we're into philosophical ground really. The truly
paranoid may never run POP-before-SMTP but a good admin can feel solid
with that kind of system installed. Heck, it's probably reasonably
secure.

--
[Simon White. vim/mutt. simon@mtds.com. GIMPS:42.88% see www.mersenne.org]
Recognizing disagreements in belief requires having enough agreements in
belief to translate or understand the words and deeds of my opponent.
-- Anthony O'Hear (combining, somewhat, several modern philosophers).
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 18 in thread
寄件者：Victor.Duchovni@morganstanley.com (Victor.Duchovni@morganstanley.com)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 16:12:05 PST

On Mon, 21 Oct 2002, Simon White wrote:

> Indeed, if a spammer already has an inkling of a dialup subnet range
> with the same domain name, it is quite likely that he will already have
> been able to send spam through an open proxy which is purely a poor SME
> connection sharing thing with some Exchange administrator who wouldn't
> even pass an MCSE... although Microsoft have made some progress in this
> direction there's a cost to upgrade unlike Open Source which offers free
> upgrades in return for a bit of clue and your time.
>

Pop before SMTP is designed to allow a user using ISP A to relay with the
mailhub of ISP B. ISP A does not need POP before SMTP to relay for their
own users, that's what permit_mynetworks is for.

The probability claims are plausible, one needs to know which third-party
dialup pool the user used to relay mail via their ISP.

Even if one can entice the user to send the attacker a message and hang
up, one then still needs to dial-in into the same IP in the dialup-pool.

If has a user account for the dialup pool, presumably one automatically
gets to relay through the mail relays of the pool provider.

--
Viktor.

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 19 in thread
寄件者："Devin L. Ganger" (devin@thecabal.org)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 18:18:03 PST

On Mon, Oct 21, 2002 at 10:42:32PM +0000, Simon White wrote:

> Anyway anecdotes aside we're into philosophical ground really. The truly
> paranoid may never run POP-before-SMTP but a good admin can feel solid
> with that kind of system installed. Heck, it's probably reasonably
> secure.

Reasonably, yes. However, security is not a magic formula. You have
to understand the strengths and weaknesses of each of the various
pieces you use so that you use them intelligently.

I didn't want statements that could have appeared to place pbfs as some
sort of magic sovereign specific to go without also pointing out that
it does, under certain conditions (likely to be found in smaller ISPs
with limited address space) have drawbacks you need to be aware of.

--
Devin L. Ganger
Co-Admin, The cabalSASL Project ( http://sasl.thecabal.org/ )
A man, a miss, a car -- a curve,
He kissed the miss and missed the curve -- Burma Shave (1948)
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 20 in thread
寄件者："Devin L. Ganger" (devin@thecabal.org)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 18:18:03 PST

On Mon, Oct 21, 2002 at 07:05:21PM -0400,
Victor.Duchovni@morganstanley.com wrote:

> Pop before SMTP is designed to allow a user using ISP A to relay with the
> mailhub of ISP B. ISP A does not need POP before SMTP to relay for their
> own users, that's what permit_mynetworks is for.

Yes, but there are ISPs that are using it for precisely that purpose,
despite the design goals because they think it's somehow more secure.

If you are one of them, be aware of the consequences.

--
Devin L. Ganger
Co-Admin, The cabalSASL Project ( http://sasl.thecabal.org/ )
A man, a miss, a car -- a curve,
He kissed the miss and missed the curve -- Burma Shave (1948)
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

寄件者：Victor.Duchovni@morganstanley.com (Victor.Duchovni@morganstanley.com)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-21 19:59:40 PST

On Mon, 21 Oct 2002, Devin L. Ganger wrote:

> On Mon, Oct 21, 2002 at 07:05:21PM -0400,
> Victor.Duchovni@morganstanley.com wrote:
>
> > Pop before SMTP is designed to allow a user using ISP A to relay with the
> > mailhub of ISP B. ISP A does not need POP before SMTP to relay for their
> > own users, that's what permit_mynetworks is for.
>
> Yes, but there are ISPs that are using it for precisely that purpose,
> despite the design goals because they think it's somehow more secure.
>
> If you are one of them, be aware of the consequences.
>

Yes, but presumably they require passwords for establishing PPP sessions
with their own dialup pool. So if the user is dialing into their dialup
pool, POP before SMTP is at least as strong as (though not much stonger
than) simply letting the user relay, and if the user dials up into someone
else's dialup pool the previous analysis holds, the spammer does not know
whose relay to try.

Either way POP before SMTP is safe.

--
Viktor.

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 22 in thread
寄件者：lst_hoe (lst_hoe@kwsoft.de)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-22 02:08:06 PST

At 22:45 21.10.2002 -0400, Victor.Duchovni@morganstanley.com wrote:
>On Mon, 21 Oct 2002, Devin L. Ganger wrote:
>
>> On Mon, Oct 21, 2002 at 07:05:21PM -0400,
>> Victor.Duchovni@morganstanley.com wrote:
>>
>> > Pop before SMTP is designed to allow a user using ISP A to relay with the
>> > mailhub of ISP B. ISP A does not need POP before SMTP to relay for their
>> > own users, that's what permit_mynetworks is for.
>>
>> Yes, but there are ISPs that are using it for precisely that purpose,
>> despite the design goals because they think it's somehow more secure.
>>
>> If you are one of them, be aware of the consequences.
>>
>
>Yes, but presumably they require passwords for establishing PPP sessions
>with their own dialup pool. So if the user is dialing into their dialup
>pool, POP before SMTP is at least as strong as (though not much stonger
>than) simply letting the user relay, and if the user dials up into someone
>else's dialup pool the previous analysis holds, the spammer does not know
>whose relay to try.
>
>Either way POP before SMTP is safe.

The only real hole i see is with (bigger) NATed Networks. If one user
fetches his mail by POP3 all others are able to relay. With a little bit of
glue they will know this and getting the provider by personal talk should
not be a problem.
But this is a clear fact of unsecure (Company-) Network, although this is
common with DSL LANs today.

Regards

--

Andreas H?dle

Kühn & Weyh Software GmbH

WWW.KWSOFT.DE

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 23 in thread
寄件者：Simon White (simon@mtds.com)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-22 02:28:39 PST

21-Oct-02 at 19:05, Victor.Duchovni@morganstanley.com (Victor.Duchovni@morganstanley.com) wrote :
> On Mon, 21 Oct 2002, Simon White wrote:
>
> > Indeed, if a spammer already has an inkling of a dialup subnet range
> > with the same domain name, it is quite likely that he will already have
> > been able to send spam through an open proxy which is purely a poor SME
> > connection sharing thing with some Exchange administrator who wouldn't
> > even pass an MCSE... although Microsoft have made some progress in this
> > direction there's a cost to upgrade unlike Open Source which offers free
> > upgrades in return for a bit of clue and your time.
> >

I am hinting, here, at the probability that a large dialup pool IP
address range is probably more interesting to scan for open relays /
proxies than for a posisble POP-before-SMTP exploit.

> Pop before SMTP is designed to allow a user using ISP A to relay with the
> mailhub of ISP B. ISP A does not need POP before SMTP to relay for their
> own users, that's what permit_mynetworks is for.

I was thinking of IP spoofing attacks in the first place; if you have to
call the ISP to get access to the right IP, you're probably in
permit_mynetworks anyway. But what if you are NATted on a large private
IP dialup subnet - with one public address which is in the
POP-before-SMTP database?

> The probability claims are plausible, one needs to know which third-party
> dialup pool the user used to relay mail via their ISP.

Exactly. This is what is reasonably hard.

> Even if one can entice the user to send the attacker a message and hang
> up, one then still needs to dial-in into the same IP in the dialup-pool.

Unless the system allows you to spoof the connections, or if you are
behind a NAT IP that is also the NAT IP of many others.

> If has a user account for the dialup pool, presumably one automatically
> gets to relay through the mail relays of the pool provider.

Precisely. That's why mynetworks should never make it into the pbsmtp
database.

--
[Simon White. vim/mutt. simon@mtds.com. GIMPS:44.31% see www.mersenne.org]
The only reason I'm burning my candle at both ends, is because I haven't
figured out how to light the middle yet.
[Linux user #170823 http://counter.li.org. Home cooked signature rotator.]
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 24 in thread
寄件者：Ralf Hildebrandt (Ralf.Hildebrandt@charite.de)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-23 07:40:19 PST

On Wed, Oct 23, 2002 at 03:58:05PM +0200, Matthias Andree wrote:

> 30 minutes is way too much. 3 might be an option. SMTP AUTH is
> implemented in most clients with GUI, so there is no compelling reason
> to use RELAY-after-POP3 instead of SMTP AUTH. Cyrus-SASL might be one.

I'd go or 10, because that's the default in Mozilla/Netscape

--
Ralf Hildebrandt Ralf.Hildebrandt@charite.de
Postfix Tips: http://www.arschkrebs.de/postfix/ Tel. +49 (0)30-450 570-155
Microsoft: A Proven Danger to National Security
http://www.infowarrior.org/articles/msdanger.pdf
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 25 in thread
寄件者：Matthias Andree (ma@dt.e-technik.uni-dortmund.de)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-23 08:20:11 PST

Bennett Todd writes:

> 2002-10-21-08:59:40 Matthias Andree:
>> pop-before-smtp is inherently insecure, no matter HOW you implement
>> it.
>
> Lightly so, yes.

Is there "lightly pregnant" as well?

> That's why there's a timeout. A default of 1/2 hour seems to be
> fine. Yes, there's a window during which someone else who got the IP
> addr of a recently-pop-authed machine could relay. No, it doesn't
> appear to be a problem in practice; I've never heard of such a setup
> actually being exploited through this timing window.
> If SMTP AUTH didn't require SASL, or if there were an implementation
> of SASL besides Cyrus, I might be more interested. Is it possible to
> implement enough SASL for SMTP AUTH without bringing in mountains of
> excruciatingly painful goo, like e.g. GSSAPI with it's ASN.1 and so
> forth?

I must amend my statement: if there is a reliable way to detect when the
client has gone offline (session termination notification,
expiry of relay permission via radius), then these points don't hold;
OTOH, if you have radius, you can omit the "POP3-before-" part.

30 minutes is way too much. 3 might be an option. SMTP AUTH is
implemented in most clients with GUI, so there is no compelling reason
to use RELAY-after-POP3 instead of SMTP AUTH. Cyrus-SASL might be one.

I pinged Brian Stafford who has a partial SASL library (LGPL) as part of
his libesmtp to see if he'd work together with Wietse to make a more
secure replacement for Cyrus-SASL, for Postfix authentication
purposes. I've not seen code yet though.

> I don't have any expire cronjobs, I just have a single persistent
> daemon that manages the data file. Works great for a single server
> that's both the pop/imap mailbox server and the smtp relay server.

OK.

> Sure it does, it does so beautifully. cdb file rebuild is so fast
> that doing a full rebuild for each update is practical in many
> settings --- as DNS hosting sites have discovered when using
> tinydns.

Well, it's ok for you, but if you have thousands (because you have a
long timeout) of clients, then it won't scale well and BerkeleyDB will
be faster.

寄件者：Bennett Todd (bet@rahul.net)
主旨：Re: [ot] tinyCDB is available in packaged form
View: Complete Thread (31 articles)
Original Format
新闻群组：mailing.postfix.users
日期：2002-10-23 13:38:03 PST

--DBIVS5p969aUjpLe
Content-Type: multipart/mixed; boundary="uAKRQypu60I7Lcqm"
Content-Disposition: inline

--uAKRQypu60I7Lcqm
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

2002-10-16-12:18:58 Bennett Todd:
> 2002-10-15-15:43:50 Michael Tokarev:
> > Bennett Todd wrote:
> > >I'd have to do some experimenting, but I really suspect
> > >pop-before-smtp would work very well indeed with cdb.
> >=20
> > One word: scalability. [...] It will work for small sites.
>=20
> Define "small".
> [...]
> Where's the crossover point? I.e. where does the cost of the CDB
> generation grow to exceed the cost of doing the DB manipulation?

Ok, in the interests of fairness, I've gotta come back and confess
that my intuition was way, way off here, and Michael Tokarev's was
right on.

While cdb can work for small sites, where the load of doing things
inefficiently doesn't make any difference anyway --- just what he
suggested --- there's no "crossover point"; no matter how small
the database, DB_Hash is faster than CDB_File for "updates" in all
circumstances.

On ext3fs:

CDBDB
107341797
1006621894
10005811372
10000133772

and on tmpfs:

CDBDB
10103406240060
10032839198009
10003513121343
1000034892536

The first column of these tables is the number of entries in the
table, where an entry is a key that looks like an IP address in
ASCII, four dotted quads randomly generated in [0..255], while the
value is "ok".

The numbers under the columns labelled CDB and DB are the number
of add/remove pair transactions completed in 30 seconds; for CDB
there's one full rebuild for each such transaction, while for DB
each such add/remove pair is surrounded by flocking, with a sync
right before the LOCK_UN. This mimics the traffic patterns seen with
pop-before-smtp, with some simplifying assumptions:-).

I used the attached program to generate that output, killing it
when the bad news was painfully obvious; rather than hassling with
emptying /tmp to overmount, I just edited the script to use /mnt for
the tmpfile for the tmpfs run.

Oh, and I ran these under Red Hat 7.3, kernel 2.4.19, on a Vaio
Picturebook C1VPK, TM5600 Crusoe CPU, 128MB RAM.

-Bennett

--uAKRQypu60I7Lcqm
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=cdbbench

#!/usr/bin/perl -w
use strict;

=head1 NAME

cdbbench --- benchmark CDB, specifically to attempt to determine its appropriateness for pop-before-smtp

=head1 SYNOPSIS

cdbbench

=head1 DESCRIPTION

cdbbench performs alternating runs, while stepping up the chosen
setsize. The setsize is intended to reflect the number of
currently-active users in the pop-before-smtp database; it's the
number of records in the database.

Each record is a random IP-address-like key (four dotted decimals,
each in [0-255]), with the value "ok".

For each setsize, starting with 10 and growing by factors of 10 (10,
100, 1000, 10000, etc). For each setsize, cdbbench generates a load
of 10 times that many random keys. These are then cycled, first
through a fresh new cdb, then through a fresh new db, as fast as
possible, for 30 seconds; at the end of 30 seconds the total number
cycled through is reported. The DB clock doesn't start until the first
setsize items have been loaded (to prime the pump); the equivalent
step means nothing with CDB since the whole CDB file must be
rewritten each pass.

Hit control-C when you get tired of waiting.

=cut

use File::Basename;

my $tmpfile = "/tmp/" . basename($0) . '.' . $$;
$│ = 1;

print "\tCDB\tDB\n";

my $setsize = 1;
while (1) {
$setsize *= 10;
print $setsize, "\t";

my @load; # in-memory cache
for (my $i = 0; $i < 10*$setsize; $i++) {
push @load, join(".", int(rand(256)), int(rand(256)), int(rand(256)), int(rand(256)));
}

# CDB
use CDB_File;
my $count = 0;
my $end = time + 30;
while (time < $end) {
my $cdb = CDB_File->new($tmpfile, "$tmpfile.tmp") or die;
for (my $i = 0; $i < $setsize; $i++) {
$cdb->insert($load[$i], "ok");
}
$cdb->finish;
my $tmp = shift(@load);
push @load, $tmp;
$count++;
}
print $count, "\t";
unlink $tmpfile;

# DB
use DB_File;
use Fcntl qw(:flock);
use vars qw(%h);
my $dbh = tie %h, 'DB_File', $tmpfile;
my $fd = $dbh->fd;
open(DB_FH, "+<&=$fd") or die "$0: cannot open $tmpfile filehandle: $!\n";
for (my $i = 0; $i < $setsize; $i++) {
$h{$load[$i]} = "ok";
}
$count = 0;
$end = time + 30;
while (time < $end) {
flock(DB_FH, LOCK_EX) or die "$0: flock LOCK_EX failed: $!\n";
delete $h{$load⁰};
$h{$load[$setsize]} = "ok";
$dbh->sync;
flock(DB_FH, LOCK_UN) or die "$0: flock LOCK_UN failed: $!\n";
my $tmp = shift(@load);
push @load, $tmp;
$count++;
}
print $count, "\n";
close DB_FH;
undef $dbh;
untie %h;
unlink $tmpfile;
}

uAKRQypu60I7Lcqm

--DBIVS5p969aUjpLe
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.7 (GNU/Linux)

iD8DBQE9twcGHZWg9mCTffwRAu0UAJ0QSJKYqBm6HYE3/ypbvOlGEfyJyACgtLa0
H9svGNdTkcm/XNn/PrLOgSA=
=LtOG
-----END PGP SIGNATURE-----

DBIVS5p969aUjpLe

寄件者：Wietse Venema (wietse@porcupine.org)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-23 14:06:04 PST

Bennett Todd:
> 2002-10-16-12:18:58 Bennett Todd:
> > 2002-10-15-15:43:50 Michael Tokarev:
> > > Bennett Todd wrote:
> > > >I'd have to do some experimenting, but I really suspect
> > > >pop-before-smtp would work very well indeed with cdb.
> > >
> > > One word: scalability. [...] It will work for small sites.
> >
> > Define "small".
> > [...]
> > Where's the crossover point? I.e. where does the cost of the CDB
> > generation grow to exceed the cost of doing the DB manipulation?
>
> Ok, in the interests of fairness, I've gotta come back and confess
> that my intuition was way, way off here, and Michael Tokarev's was
> right on.
>
> While cdb can work for small sites, where the load of doing things
> inefficiently doesn't make any difference anyway --- just what he
> suggested --- there's no "crossover point"; no matter how small
> the database, DB_Hash is faster than CDB_File for "updates" in all
> circumstances.

It may still be fast enough for small sites. With up to 1000
pop-before-smtp entries in a table, the cost of an update is only
1-2 milliseconds. On a system that is maintained by a private
person the impact would be negligible.

Thanks for measuring this.

Wietse
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 28 in thread
寄件者：Rahul Dhesi (dhesi@rahul.net)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-23 15:30:07 PST

On Wed, Oct 23, 2002 at 04:31:02PM -0400, Bennett Todd wrote:

> While cdb can work for small sites, where the load of doing things
> inefficiently doesn't make any difference anyway --- just what he
> suggested --- there's no "crossover point"; no matter how small
> the database, DB_Hash is faster than CDB_File for "updates" in all
> circumstances.

If I were designing a pop-before-smtp system from scratch, I would do
it like this:

1. A function that creates (or updates the timestamp of) a file
representing an IP address. The filename is generated by replacing the
first N dots in the IP address with slashes. N is in the range 0 .. 3.
Number of system calls needed: two (an open and a close).

2. A function that returns true or false according as the file for a
given IP address exists or not. Number of system calls needed: one (a
stat).

3. Every half hour a cron job does this:

cd somedir; find . -type f -cmin +30 -print │ xargs rm -f

This is how I first implemented pop-before-smtp in 1997. I used N = 0.

As a possible optimization, the daemon that creates the files can
consult an in-memory hash table and avoid recreating/retouching the same
file more than once every 20 minutes.

As another optimization, we can run the cron job every 24 hours instead,
and have the lookup function (in item 2) return true only if the file is
no older than 30 minutes. This increases the number of system calls to
two: a gettimeofday and a stat.

I'm guessing that the reason we are discussing the efficiency of
different types of databases, instead of using the simple and efficient
scheme I have described above, is because Postfix's smtp server does not
have a built-in facility to look for a filename representing an IP
address. Thus I believe the cart is dragging the horse here.

Rahul
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 29 in thread
寄件者：Wietse Venema (wietse@porcupine.org)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-23 15:34:28 PST

Wietse Venema:
> Bennett Todd:
> > 2002-10-16-12:18:58 Bennett Todd:
> > > 2002-10-15-15:43:50 Michael Tokarev:
> > > > Bennett Todd wrote:
> > > > >I'd have to do some experimenting, but I really suspect
> > > > >pop-before-smtp would work very well indeed with cdb.
> > > >
> > > > One word: scalability. [...] It will work for small sites.
> > >
> > > Define "small".
> > > [...]
> > > Where's the crossover point? I.e. where does the cost of the CDB
> > > generation grow to exceed the cost of doing the DB manipulation?
> >
> > Ok, in the interests of fairness, I've gotta come back and confess
> > that my intuition was way, way off here, and Michael Tokarev's was
> > right on.
> >
> > While cdb can work for small sites, where the load of doing things
> > inefficiently doesn't make any difference anyway --- just what he
> > suggested --- there's no "crossover point"; no matter how small
> > the database, DB_Hash is faster than CDB_File for "updates" in all
> > circumstances.
>
> It may still be fast enough for small sites. With up to 1000
> pop-before-smtp entries in a table, the cost of an update is only
> 1-2 milliseconds. On a system that is maintained by a private
> person the impact would be negligible.

Argh, that should be 30-60 milliseconds.

> Thanks for measuring this.
>
> Wietse
> -
> To unsubscribe, send mail to majordomo@postfix.org with content
> (not subject): unsubscribe postfix-users
>
>

-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

Message 30 in thread
寄件者：Matthias Andree (ma@dt.e-technik.uni-dortmund.de)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-24 07:10:04 PST

Rahul Dhesi writes:

> I'm guessing that the reason we are discussing the efficiency of
> different types of databases, instead of using the simple and efficient
> scheme I have described above, is because Postfix's smtp server does not
> have a built-in facility to look for a filename representing an IP
> address. Thus I believe the cart is dragging the horse here.

You can get all this for free with SMTP AUTH. No database required
except that one which holds the password and has a few writes a day
only, even at big sites. No file system sweeps necessary to expire old
relay permit files, and no cache to avoid touching the file system too
often.

--
Matthias Andree
-
To unsubscribe, send mail to majordomo@postfix.org with content
(not subject): unsubscribe postfix-users
Post a follow-up to this message

寄件者：Matthias Andree (ma@dt.e-technik.uni-dortmund.de)
主旨：Re: [ot] tinyCDB is available in packaged form

View this article only
新闻群组：mailing.postfix.users
日期：2002-10-28 02:54:08 PST

"Peter H. Coffin" writes:

>> You can get all this for free with SMTP AUTH. No database required
>> except that one which holds the password and has a few writes a day
>> only, even at big sites. No file system sweeps necessary to expire old
>> relay permit files, and no cache to avoid touching the file system too
>> often.
>
> Unless setup and tuning time costs nothing, based on the number of
> difficulties people have setting SASL up, you have a different
> definition of "free" than I do.

"free" here means "free of additional system load".

I've been using SASL1 with the pwcheck daemon to authenticate users
against /etc/shadow or with sasldb without difficulties. When using the
pwcheck daemon, make sure the permissions to /var/pwcheck are right, I'm
using

drwx--x--- 2 root postfix 58 Sep 26 23:30 /var/pwcheck/

Note group ownership and execute permission (That's chown root:postfix
and then chmod 0710.)

I'm aware that this scheme has been superseded in the meanwhile, but it
still works with the latest Cyrus-SASL v1 version. It is disputable
whether this is safe enough to the local machine -- you get to balance
"local security" (use POP3-before-SMTP) against "network security" (use
SASL for SMTP AUTH).

--
Matthias Andree