[Xitami] Sub directory access

[Ewen McNeill]

In message <997380321@coconutcomm.com>, john.uhlig@coconutcomm.com writes:
>Hmm, maybe that is the problem - the "index" listing of all user pages are
>created dynamically for every request, depending on the user input, i.e.
>searching a particular region. I *assumed* that if I submitted the TLD to
>search engines, that it would automatically get all subdirectories under that
>domain. So what you are saying is that there should be a hard coded index page
>with links to all of the directories/pages under the TLD?

Search engine spiders (the things that index pages) work by getting a
page (eg, the top level page) and then following the links in that, and
the links in the pages that gets to and the links in the pages that gets
to and...

If you've got a bunch of directories that _are_ links to from the index
page at the front, it'll find them fairly easily.  If you've got a bunch
of pages which aren't linked from there (and/or aren't linked from
anywhere but CGI scripts, which generally search engines will avoid
running) then it won't be able to find them, since it doesn't know to
look.

As someone else said, either rearrange your site so that there's a set
of links that leads there, or submit the URLs separately.

If a web browser can access the pages, so can an indexing spider.  The
question is "does it know that it should"?

Ewen

-- 
Ewen McNeill, Technical Consultant, iMatix Corporation  www.imatix.com