@Mojeek I am in the process of adding Mojeek as the default search engine on Privacy Browser, but a user informed me that searches involving Cyrillic characters return a 403 - Forbidden error. Is that intentional?
@privacybrowser Hey Soren, we sent you an email also, but for anyone with keen eyes on this, here's an excerpt:
"At the moment we don't offer Cyrillic searches as our index does not currently contain crawled results which would provide anything useful. It takes time to build up an index and for the moment we are focussed on improving our results in English & some other (mainly Romance) languages. It's likely the Cyrillic characters return a 403, due to our bot-capturing software."
@Mojeek @older You said previously that requests with Cyrillic characters are classified as bots (in my experience 100% of the time). You also said that you have no indexes for Cyrillic characters, so searches for them always return no results. So, why not just return a page saying there are no results instead of returning a 403 - Forbidden error? It has the same impact on your system, while being much more helpful to legitimate searches from real people.
@Mojeek @older The text of the email you sent me is published at https://www.stoutner.com/switching-from-startpage-to-mojeek/ . In it, you say that it may be the problem, but that you tried several searches with Cyrillic characters that were not blocked. However, my experience is that all English queries produce a webpage, even if no results are found, and all Cyrillic queries produce a 403 - Forbidden error. Were you perhaps testing from an IP address that is whitelisted from your bot blocking tech?
@privacybrowser @older Business model explained here: https://blog.mojeek.com/2020/12/frequently-asked-questions-about-mojeek-business-model-surveillance-privacy.html We have no-tracking Ads now with a closed set paying customers: https://www.mojeek.com/support/ads/
@older @privacybrowser Indeed. But every character is prefixed with %D0 and that was a signal showing up in bot attacks. We might have been overzealous in our defences, or even messed up. As you saw and pointed out it was killing human queries. Anyway, apologies about that and thanks for highlighting the issue.
@Mojeek A user pointed out that searching for `site:stoutner.com "privacy browser"` always produces an `403 - Forbidden` error, while searching for `"privacy browser" site:stoutner.com` does not.
@Mojeek The issue was originally reported at https://forum.f-droid.org/t/privacy-browser-search-engine/13408/19 The reporter experiences it intermittently using Tor, but I have experienced it consistently (at least so far) without using Tor or any other VPN.
@privacybrowser Hey Soren, this should not return a 403 anymore: https://www.mojeek.com/search?q=site%3Astoutner.com+%22privacy+browser%22
Fosstodon is an English speaking Mastodon instance that is open to anyone who is interested in technology; particularly free & open source software.