Skip Navigation

User banner
Posts
58
Comments
1,442
Joined
2 yr. ago

  • Unfortunately, this tracker will need to rely on centralized databases such as MAL, anidb and anilist as I can't really seem to find an easy way to curate new animes without them.

    1. I am not very sure about this point honestly, as previously discussed in this thread, having a database that is only maintained by the community has its own disadvantages that definitely outweight its advantages. For example, you would need constant moderator approval, which is slow.
    2. I am not sure what you mean by the ability to share them privately. Do you mean like whitelisting certain people that can view your animelist? Like how MAL has a friends only thing?
  • Yep, that's what I was thinking.

  • Oh, I see. Ok, I get it now. JSON indeed would be very slow for this project. I also do think of maybe caching some stuff for this reason.

  • What? What do you mean full-fledged JSON. I won't be storing the animes in JSON, but in a PostgreSQL DB. I don't understand what you mean...

  • Python, Rust and Lua. SQL if that counts as well.

  • What do you mean by "lists in the code"? Which lists and do you mean by "in the code" hardcoded?

  • Oh well, no. The database won't be decentralized. That just invites chaos. Bookwyrm did that and now there are lots of duplicates in their library, which I definitely want to avoid. The things that will be federated are animes' forums (as communities in Lemmy for example), and those communities will have threads that will be either a forum thread or a review which others can comment on from any federated platform. Some other things might get federated in the future.

    I decided to just scrape either MAL alone or multiple sources.

    If you have any ideas on how the database can be decentralized while efficiently avoiding duplicates and spam, please do say it.

  • PMs today aren't really private as in E2EE. They're private as in only you, the receiver and the "trust me bro" company can see them. In special data breach cases, the attacker as well.

  • shitpost of the day

  • I can see that being a problem.

  • Correct. I thought there was some way to prevent those duplicates because of the federation, but apparently there's no such check being done so it's all duplicates. For that reason I won't put any effort into making the database federated. Only threads, reviews and comments will be federated. This might change as I make it though. There are also clubs in MAL which I might copy and implement in this project and they could be federated as well.

  • Wouldn't Internet Archive be a bit slow? And also, I don't want to stress their servers.

  • That seems like a good idea. I'll keep it in mind.

  • Yeah, though I am not sure how the federated instance admins would react. I am planning for every anime, manga, vn, etc. to have their own communities. This means about over 100k communities being made in an instant. Maybe instead of creating the communities in an instant, creating them when user activity first happens would be more fit. But this would also restrict other platforms' users being able to comment on never heard or new anime entries until someone from the anime tracking platform comments or reviews them.

    Thanks btw.

  • It is, but I have no choice other than that if I can't scrape websites.

  • I see. I can use the Jikan API to scrape animes and mangas which will take approximately 1-2 days after I get approval then. Oh and I forgot to mention, the federation part isn't really how people think it will be I guess. The only federation that will be done will be the reviews, threads and the comments in them. With every anime/manga/vn, etc. being a new community which will contain those threads and reviews. And because of that, I don't really know if this project is something people would want to self host. I guess I could provide full dumps of the database every month or something but I suppose that would be expensive. Then there are images as well, which will take hundreds of GBs easily even in compressed form.

  • Guess I can download and host them in the EU as this project won't be commercial.

  • Ok cool, so even when that is out of the way, are the images copyrighted? Because I'd like to download the images and host them on my servers.