[maemo2midgard] 'Maemo.org forbids GoogleBot'

[Niels Breet]

> Hi,
> http://blog.haerwu.biz/2008/04/10/the-curse-of-maemo-closed-source-compone
> nts/ says at the end: "BTW: I wonder why Maemo.org forbids GoogleBot… It
> is really hard tofind Maemo related things with Google :("

robots.txt:


User-agent: Mediapartners-Google*
Disallow:

User-agent: *
Disallow: /test/

This means we only block the adsense/adwords crawler for google. This
doesn't matter, because we don't have google ads.

And all bots are disallowed to crawl /test/, which doesn't effect the site
crawling at all.

http://www.google.com/search?q=site:maemo.org+the+-garage+-bugs+-stage+-repository&hl=nl&c2coff=1&start=10&sa=N

At least 3370 pages index for maemo.org itself. The site is properly
indexed by google.

The writer of that blog post didn't look close enough and made a statement
that was not true.

> Any idea or opinion? I'm mentioning this because search related issues
> are still part  of our current sprint.

Our 'internal' search box has problems with garage and lists.m.o. It
searches http://maemo.org just fine.

> Quim

- Niels