How to search: the sublime art
There are 350 million sites
out there, doubling every four months... add to this the 'second' internet
(the new 'university connection' net)
and the wide and huge usenet, and you already have a plethora of universes
to explore. And there are also
all the old dark web-corridors, made of forgotten archies (and veronikas! :-),
and much more... Therefore: where, where, where is the info you need?
Study! If you don't master the sublime art of
searching you'll never find that what you seek!
You'll be able to learn here the first elements of this art. Once you'll have understood them, you'll be
able to go
even further (pretty quickly) if you learn perl (THE language for bots building) and
if you will study and research a LOT. Meanwhile, for
a start, you will be able to peruse this 'heavy' search engines page (which loads quickly
anyway, since it does not
carry silly backgrounds, useless frames or slow loading futile
publicity). There is also a
"light" version, which you may want to copy
onto your hard disk as a quick search starting point.
Since search engines do play a relevant role when searching (duh), I have also added a
search engines' vagaries page, that you may find interesting.
Of course, you should learn
to search the web using inter alia my own lessons!
Special 'Mover' for this search engines ('heavy') page
Use always lowercase search strings: fravia will fetch "fravia" and "Fravia";
frAvia will fetch only "frAvia" occurrences.
2) keep in mind that searching on the web is THE CONTRARY of a search in a normal
library: in a library, the more generic your search, the more easy it is to find
what you are seeking, on the web the more specific your search, the more easy it is to
find what you are seeking. So it is relatively easy to re-find a page that you happen to have read, even
if it has changed site, as long as you remember (well) a part of it. Try it: just cut and paste into Altavista these words (from my main page)
"contains many teachings, and will help you gain knowledge that you will not
find elsewhere. Please wander slowly inside: sip a good" (Yes, all of the preceding 'blue' words:
don't forget the " and remember that there's no limit at
what you can paste inside a search engine form)... you'll find -amazingly enough- my very page!
3) It helps to do lots of searches from lots of different search engines,
each of the big engines is different, and each has its own
For instance, Yahoo does things in categories, Alta
Vista uses phrases best, and Lycos is good for huge keyword searches. Webcrawler gives
at times surprisingly good results.
Hotbot has a HUGE database, and is more than good
for just quickie searches, anyway as it seems: Altavista and Hotbot are 'going down', while
'catching up'... Hotbot is easyly configurated to avoid 'stale' links... well, yes, there are
quite a lot
of "must know" differences between all the main search engines.
I still prefer Altavista though... my advice?
Use always (at least) TWO search engines (Infoseek and Altavista IMO) and a directory!
According to myself (and that's not a bad source of info :-) combined
searches using multiple engines increase the
likelihood of finding the desired information by 4-5 times. A very good idea is also to
use one of the existing
inference robots and scripts...
4) DO NOT GET DISTRACTED! Web surfing is terribly distracting! One search session out of three
crashes before reaching the desired target because you got lost on those 'interesting
side-trips' that you found on your way (translated,
this means that you moronically just forgot what you originally wanted and began following
godzilla knows which
paths). Be coeherent (and be 'hard', man: real seekers are quite concentrated fellows) write down your target and go for it (and NOTHING else until you
find it!) Devote a minimum of 30 minutes and a maximum of 60 to your search session, that's the
best time range IMO, less isn't enough and more bores. Remember also that real queries are made in
phases: phase one is 'broad', after a while (say a week, you must first digest and understand
what you found) you go for a more concentrated approach, and then, last phase, you really find
what you wanted (after having perused the newsgroups and searched the archies).
5) Search USENET! Remember that there are THREE search strategies and that usenet searching is
the second and very powerful one!
ONE: you search yourself (searching);
TWO: you use people that have already searched (combing), for instance on usenet, and use the
links and info that they have gathered during years-long queries (no need to re-invent the wheel
THREE: you look what kind of people (and from where) come and sniff at the info
you provide on your own sites (klebing)... but that's advanced stuff and you'll learn it
in due time...
6) I would recommend that you turn off auto image loading
before submitting a request to ANY search engine when you'll use the forms below,
this will speed up your searches CONSIDERABLY. (Come to think of it, it's always better to turn off mage
loading IN GENERAL, unless you really need to find/see images). Btw, the control about
image loading options is another of the many fields where the new slick Opera browser
demonstrates its absolute superiority over both browsersaurii...
AltaVista is USEFUL: it's the largest Web index, with millions
of Web pages AND articles from usenet newgroups (and these may be REALLY useful). Altavista
is still the search engine of choice for all "old hands" of the web, yet Infoseek is now (1998)
slowly catching up!
AltaVista can also be used to get all the page linked to your page.
I can get with the following link the pages linked to a no more existing page of mine (useful in order to find stale links):
try it. Another
useful method is to use the *HTML TAGS*, if you for instance search "image:bettie", AltaVista will find quite a lot of images
about Bettie Page.
Search Method: Keyword Data Base System: Full text, largest and more inclusive indices Operators: If used without suitable operators Altavista produce
enormous noise and little signal
For the more paranoids (or the more careful) among you, here is a link to
search form (Courtesy of fravia... do not leave your tracks around!)
Yahoo (it's an attempt to catalogue
the entire Web, search on a topic) Very good for beginners, well organized.
Search Method: Subject and Keyword Data Base System: Limited coverage: indexed by human operators Operators: In keyword searches selects only sites that contain
ALL search words. If no exact match is found switches automatically to AltaVista
Exite (a concept-based search engine,
tries to figure out what you mean). It's awfully slow, because they want
to "impose" you to read the stupid pubs before getting on with your search. Use
it only as last resource or if you are a beginner (it's very easy).
Search Method: Subject and Keyword Data Base System: Full text, circa 50 millions docs, titles are not
searched (or so it seems) Operators: Automatic word-root search (without the need to use a joker like '*')
and sorting by site
WWW Worm: It is extremely flexible, allowing regular expression searches on URL, subject
and content. In spite of its flexibility, it's not difficult to use, and there exist a
Lykos Big and slow, Lykos has a large number
of binary files in its database, has greater depth than most search engines,
because it also indexes FTP archives and Gopher menus. Besides, you can search with Lykos
per email. There are some other interesting Lykos
resources as well, including a list of the frequency of over six million words used
on the Web. Lycos is one of those facilities that's almost too good, presenting you with
more information than you really need, but it's a great resource if you use it carefully to
narrow down your search.
Webcrawler fast and cool, returns surprisingly relevant results. This excellent search engine indexes the contents of Web documents, so
you can find pages that contain a particular word or phrase.
InfoSeek Net Search Good output, their attempt to get "commercial" failed against the sound Web altruistic spirit (:-), it's now a free
service. This claims to be the largest set of searchable
indexes to WWW pages and to USENET newsgroups. 1998: INFOSEEK is getting better
and better! (They value QUALITY very high!)
Search Method:Subject and Keyword Data Base System: Full text, over 50 million pages Operators: detailed instructions under
help HotBot (ex Inktomi, claims to have the
largest index and the best scalability in terms of keeping up with the Web's
exponential growth, claims to be able to re-index the entire Web every week)
Well known for its awful colors, this was one of the BEST search engines available,
unfortunately slowed down by useless graphics frills.
Search Method: Keyword Data Base System: Full text, over 60 millions documents Operators: Simple and advanced... detailed instructions under
Internet search wizard Compuserve's forms' selection, this one you are using is better
ftp search 3000 FTP sites
around the world, quite slow and busy at times. ("This server is located in
Archie request form (but you better use Archie trough email,
see here how to do it).
I have decided to use this funny name for those scripts that allow you to query (almost)
simultaneously more than one search engine. I have two forms here for you:
inference find and dogpile. Both are mighty interesting for the casual or the
'hurried' searcher, yet I believe
inference find to be a VERY USEFUL TOOL even for advanced seekers:
It will not only query AltaVista, Excite,
Webcrawler (quite a good and correctly limited choice per se), but it will present to you SURPRISING RELEVANT ANSWERS in a
special formatted *.htm file that you can IMMEDIATELY DOWNLOAD AND USE!. Here you are:
Here is how you perform an e-mail search
on dogpile... Infact, this is the above form "cracked". Dogpile
searches ALL main search engines and compacts the results for you. Often e-mail searches are cleverer than
live searches on the web (for the same reasons that make "dead listing" more effective than live winice approach when
reverse engineering, btw :-)
email an Agora server (be ashamed if you do not know what
an Agora robot is) with following TEXT (if you want to search for
"numega" AND "softice" for instance :-)
Really useful search engines allow you to check which strings others are
using as queries. We have already seen (inside the "klebing" search technique section) how
important "alien" search strings are, for each one, in order to ameliorate your own
search strategies... yet it's
still pretty funny to check what people look for... (and pretty sad at times :-(
see how frequently people misspell their queries, and how incredibly
often strings like 'Pamela Anderson'
or analoguous idiotical "slave lemmings" subjects get search requests.
use a "professional" search page on your harddisk!
You should copy the "professional" ("light") version of the search engines' forms on this
same page on your
hard disk and choose it as "bookmarked" or "hot" (or as "favourite" if you
are a Micro$oft's slave). Use it as your
"main search engines" starting page.
You'll have quite a lot of advantages:
1) You'll have all main search engines at your fingertips
2) The search starting page won't be fetched from a busy server and will therefore load immediately
3) The search starting page won't have any images (and therefore don't suffer ANY delay whatsoever)
4) The search starting page will spare you HOURS of connection wasted time!
5) The search starting page will be edited by YOU with your own additions and comments! (send me any interesting addition :-)