CuratedCrawl did it's first search (ᗒᗨᗕ) Alpha release, future plans and possible full release date

Written on 17th June 2023 at 12:50 PM

AYO THE FIRST RELEASE IS HERE!

That's right the CuratedCrawl search engine just did its first successful search! The CLI client is so close to being production ready and will probably be pushed to the development branch in approximately 3 days!

What is done so far

Well even though there is still a lot to clean up (like missing documentation, unorganized code, etc.) I think the scraping and discovery infrastructure is ready for the first beta release!

This project was supposed to be a lot easier... There has been a lot of rebasing and starting over again which ultimately made everything much harder than it was.

The website is still not ready, like, there's not a single line of code regarding the web page which will actually serve the results and provide a user-friendly way to use the search engine. It shouldn't be hard to create though since I already got a proof of the concept version working.

I'm thinking of designing something like this as the web UI

Possible release date

I can't say anything for sure but as far as I can guess it should be ready soon™

I need your help!

Now that Reddit has been burning since the last week, I don't know where to promote my search engine. Tumblr seems to be interested in neocities stuff though I don't know which tags I should be posting it in.

Another problem is preventing DoS attacks which can be done by spamming search queries and thus overwhelming the server. A solution I can think of is putting a captcha every 10 searches though that seems like a really user-hostile option.

If you know where to promote CuratedCrawl or if you have solutions to the DoS problem email me at ae@aesistril.com