Re: Scalable Mail solution with NAS

[Adrian Chadd <adrian () creative net au>]

On Wed, Jan 31, 2001, Neil J. McRae wrote:
> > > But, with 200k mailboxes, you should have an automated way to do that anyway.
> >
> > Hah. Unlink the directory, and do a background fsck every few hours? :)
>
> I don't know why you'd want to do the above, but you could add code
> to the deliver agents; When inbound email hits the system: create
> the require directories and files if required, this could be
> mail.local, deliver or something similar.  When mailbox is emptied
> get the delivery agent [could be pop3d or imapd] to delete any
> empty directories, then growing directories can be kept under
> control.

oh, I was joking about the above. Yes, you're right.

> > The trouble with the above format is that you're ignoring any locality
> > that exists in the filesystem. For example, in Berkeley FFS, files in
> > a given directory are allocated in the same cylinder group (or at least
> > it is attempted..)
> >
> > Which, under heavy heavy load could actually give a slight performance
> > boost on a non-filled FFS.
>
> Agreed, but depending on the scale you'd most likely want logging file
> systems otherwise reboots could be painful.

Uhm, I didn't think I was going to, but I guess its time for a plug.

I modified FFS to remove its namespace and place a flat inode-based
namespace in its place.

Its called IFS, and it can be found in FreeBSD-current.
Directory operations are done with inode numbers. One of the things
on my todo list is to pass "locality information" in with the
create() (ie, say "be close to inode <foo>").

fsck'ing an IFS partition is fast, because it doesn't need to check
the pathname tree.

So, there's no reason you need to try to do tricks with the
UNIX directory namespace. You'd be surprised how much RAM/many
diskops are wasted in doing lookups and attempting to cache them..

(and thats my last public post on this topic.)



Adrian


> Regards,
> Neil.

-- 
Adrian Chadd                    "Sex Change: a simple job of outside 
<adrian () creative net au>       to inside plumbing."
                                    - Some random movie