I've always wondered how it's possible. No way they've got some crazy software optimisations that nobody else can replicate right? They've gotta just be throwing a ridiculous amount of compute power at every request?
I said it in a higher comment with other info but try looking up a remote community that isn't already known by an instance, without being logged in. It won't look it up for you and just silently fail. If unwanted content is what you're worried about unfortunately a malicious actor can basically just drop content directly into your instance without prior notice if your federation is open. This is why db0 is working on systems that will in the future work like shared blacklists (opt-in of course).
Anonymous users can't actually lookup other instance communities through yours in the same way logged in users can. They'll only be able to see a remote community if a user who's already been logged in on your instance has searched it up before and/or is subscribed, but they can't just arbitrarily make your instance look up other instance communities.
Then I guess you could configure nginx to not allow /c/ requests that have an @ unless the "jwt" cookie is present and do the same with your search endpoints. Of course, someone could just add an arbitrary jwt cookie to try and bypass it but if the point is more to make the average anon user not waste your server resources I think that should do. Without search and without the communities visible via /c/ everything within it wouldn't be indexed in search results so the only way for them to see a federated post through your instance would be a direct link to one.
Why are people downvoting this? It's just practical. If you don't reduce the amount of storage you use, the cost of storing will constantly and steadily rise, will donations or personal budgets keep up? Maybe on lemmy.world or other mega instances but it just won't be the case for everybody. I think purging old content is gonna be a reality eventually, even if it takes a really long time before it catches up to the larger instances. And it's going to be OK as long as, as this person suggested, the rules for purging old stuff is tenable for everybody.
For example, does lemmy.world, lemm.ee, and sh.itjust.works really NEED to keep each other's entire federated post history, in perpetuity? As these guys grow larger wouldn't it make sense to start purging very old duplicate content between them? Stuff that hasn't been accessed on the instance in, say, over a year? Mind you, I believe that before we get to this point, there will be other systems in place. For example, the Reddit archive sites were never run by Reddit, and they often contained ads or other monetization strategies. Donations can keep the most recent or relevant content up on the instances, but somebody somewhere is gonna have to pay for this content to stay out there. For all we know, it's gonna be fucking Google and their seemingly unlimited cache. For all we know, some person at Google is spending his 20% personal project time subscribing a bot to everything on the fediverse and collecting data for some kind of new search engine right this very second on Google's hardware.
Just wanna remind everyone that Mastodon also has a code of conduct, just like Lemmy does. The Taliban are explicitly not welcome on Mastodon as a platform. Nobody's gonna be able to stop them short of defederating, but just thought I'd share this.
The two main Devs of Lemmy do this full time. They're not hired in a traditional sense, but the project is funded enough for them both to work on it as their full time job. Now, this isn't a problem with open source, I'm a professional software Dev and you would not BELIEVE how many enterprise, proprietary systems are still doing things like building SQL statements by directly concatening strings that come from user input (especially in enterprise software cause, well, who's gonna fuck around with it?). No, this is a problem of having this many eyeballs on you. The tiny little places they slipped up and didn't properly sanitize a user input string was found and exploited. Most proprietary systems do NOT reach this level of user count, and in particular Lemmy attracts a certain more tech-savvy demographic that would've found this sooner or later, malicious or not. Remember, this vulnerability was not just found, somebody was looking for it.
I think in that case the "forced" centralisation is purely constructed. There are no mechanisms preventing somebody from creating an android community in their own instance and federating it with lemmy.world. Even if !android@lemmy.world is permanently locked, that fact isn't really a barrier to entry for another android community to pop up, just that that community was able to establish a subscriber base over time but I don't see why another android community couldn't do the same given some time, especially if the available android communities at the moment are locked and restricted.
Basically nothing of unique value really comes from Reddit the company. Their platform is easily replicable (as you can see from the MANY Reddit clones other than Lemmy) and their staff are glorified powermods and repost bots. The only thing Reddit has of value is its communities, and those got where they are despite Reddit's best efforts.
Communities are Actors in AP with type "Group", and in Lemmy they're automated to Announce anything posted to them to all followers. Users are also Actors with type "User". I think the word Actor is intentionally used in the spec to avoid people thinking they're meant to represent actual people. In order for us to see elonjet here the account would have to either tag a Lemmy community or reply directly to a Lemmy user's comment.
And even when problems are found, like the heartbleed bug in OpenSSL, they're way more likely to just be fixed and update rather than, oh I dunno, ignored and compromise everybody's security because fixing it would cost more and nobody knows about it anyway. Bodo Moller and Adam Langley fixed the heartbleed bug for free.
Because they're all different applications. I think the confusion here is between ActivityPub the protocol, and the applications that actually use it. The applications that use AP to federate are all different, they have different data structures, hell mastodon and Lemmy/kbin are completely different at a conceptual level. They just communicate with each other via AP, but once they receive the AP message they convert it into their own data structures and concepts. And you should note that AP is technically a communications protocol, it doesn't prescribe how stuff should be stored or sorted after an object is communicated between two servers and doesn't really prescribe a way to browse through the historical activities of a person. These are things implemented by the application you're using. So it's not like you could just write an app that combs through all that data available on the fediverse, you'd need an instance that federates with all these places, then an app that uses that instance. Technically feasible but so far nobody's done it yet, but you can see how some people reply to threads on Lemmy via Mastodon.
How can they compete in a space where people are already there providing a service without trying to extract value from them?
Why would one of these larger instances sell out when their userbase can sustain them and selling out is antithetical to the reason they started the instance in the first place?
I and many people like me would be fine in our own instances. We'd just defederate. If, say, lemmy.world sold out those guys would just have to switch instances. It's a pain, yes, but it's possible.
These guys aren't happy with some support. They want all the support i.e. money. Feels like no tech corporation thinks about its products long term anymore. Just the most readily available cash grabs possible, even if it means possibly losing future revenue.
That's the other thing, people wouldn't do that if it didn't get them karma every single time. Karma isn't publicly displayed on Lemmy, and if the vision of very spread out communities comes to fruition it also just wouldn't make sense anyway.
I've always wondered how it's possible. No way they've got some crazy software optimisations that nobody else can replicate right? They've gotta just be throwing a ridiculous amount of compute power at every request?