[Xitami] Xitami performance regarding directory structures

[Michael diLorenzo]

To second what Gene said, NTFS can handle massive numbers of files per  
directory.  In fact, I'm not sure that there is a limit to the number of  
files per folder.  Technically, the maximum number of files on a single  
NTFS volume is over 16 billion billion, although WindowsXP imposes a limit  
of just over 4 billion.

I think your issue is going to be performance of NTFS rather than quantity  
of files.  Microsoft recommends disabling short-file name generation if  
you're going to have more than 300,000 files per folder on an NTFS  
volume.  They provide some information and instruction for this at  
http://www.microsoft.com/resources/documentation/Windows/XP/all/reskit/en-us/prkc_fil_tdrn.asp

I would venture to say that your Xitami performance is only going to be  
limited by the performance of NTFS.

Michael

On Wed, 08 Jun 2005 23:26:26 -0400, Gene Angel  
<fuzzydog@whiteshepherd.net> wrote:

> RPO wrote:
>
>> I am running W2K Server (NTFS) and I have done some research concerning  
>> directory limitations within the W2K environment and now I am looking  
>> for potential limitations with Xitami.  My server will host potentially  
>> 10s of thousands  of users.  Each user will have the ability to create  
>> their own photo albums and then have photo books within each album.   
>> From a storage standpoint each user will have their own user directory,  
>> that user directory will hold from 1 to 15 sub directories that will  
>> represent the photo albums.  Then each photo album directory will have  
>> from 1 to 15 photo book albums.  Sorry for the extensive path below put  
>> I wanted to give an accurate representation of what I am looking to  
>> do.  The directory tree will look like this (each numbered directory,  
>> ie: 1000, 2000 - will hold up to 1000 user directories each).  The  
>> things I am considering are sub-directory limitations per parent  
>> directory and performance issues for Windows considering the directory  
>> structure and then ultimately Xitami's performance handling the  
>> directory structure.  Comments?!
>>  This is the only info I found pertaining to NTFS limitations:  
>> http://www.microsoft.com/resources/documentation/Windows/XP/all/reskit/en-us/Default.asp?url=/resources/documentation/Windows/XP/all/reskit/en-us/prkc_fil_tdrn.asp  
>> but I am uncertain if I fully understand its implications.
>>  xitami
>>      - webpages
>>           - albums
>>                - 1000
>>                     - user1
>>                          - photoAlbum1
>>                               -photoAlbum1_book1
>>                               -photoAlbum1_book2
>>                          - photoAlbum2
>>                               -photoAlbum2_book1
>>                               -photoAlbum2_book2
>>                     - user2
>>                          - photoAlbum1
>>                               -photoAlbum1_book1
>>                               -photoAlbum1_book2
>>                          - photoAlbum2
>>                               -photoAlbum2_book1
>>                               -photoAlbum2_book2
>>                     - user3
>>                     ...
>>                     - user1000
>>                - 2000
>>                     - user1001
>>                          - photoAlbum1
>>                               -photoAlbum1_book1
>>                               -photoAlbum1_book2
>>                          - photoAlbum2
>>                               -photoAlbum2_book1
>>                               -photoAlbum2_book2
>>                     - user1002
>>                          - photoAlbum1
>>                               -photoAlbum1_book1
>>                               -photoAlbum1_book2
>>                          - photoAlbum2
>>                               -photoAlbum2_book1
>>                               -photoAlbum2_book2
>>                     - user1003
>>                     ...
>>                     - user2000
>>                - 3000
>>                - 4000
>>                - 5000 (5,000 users)
>>                ..
>>                ..
>>                - 100000 (100,000 users)
>>                ..
>>                ..
>>                - 1000000 (1 million users)
>
>
> I hold  alot of users/folders under Win2k3 NTFS.  NTFS uses a btree (I  
> believe) so it's pretty good on bulky directories.  You can increase  
> performance two ways through windows.  Use a directory tree index scheme  
> to limit your folder/files per directory or (and I think the bigest  
> performance gain for you) would be windows disk Cache.  Windows disk  
> cache comes from "unused" memory so the more free memory you got the  
> more file/directory tables will be able to be held in memory.  This lets  
> the system bypass hard drive tables and access directly from RAM which  
> is much faster.  My main server has 1 gig of ram and I will upgrade it  
> to 2 before long.  Win2k3 also has faster file access then win2k.
>
>   Gene
>
>
> --
> Xitami Users Mailing List -- For Xitami support
> To unsubscribe: http://lists.xitami.org/mailman/listinfo/xitami



-- 
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/