Hey there.
Post that are made to the lightnovels community (!lightnovels@ani.social) from the lemmy.world instance (for example this post https://lemmy.world/post/13768303) don’t show up here.
It shows up now that I manually searched for it. Ani.Social appears in “lagging instances” (along with 61 other instances) for lemmy.world.
I just checked the Lemmy Matrix chatroom and the Lemmy.World admins are aware of it already. Hopefully they can fix it soon.
Thanks. The post do show up now.
So, I noticed this earlier today and started lurking in some matrix channels to try to figure out what was going on. From what I can gather, there seems to be some issues with kbin spamming lemmy.world with activities (tons of up/down votes looping over and over). This led to a huge influx of database interactions that then cascaded to every other lemmy server that world federates with. This continued for some time until the world admins put in place some bans and filtering on their proxy to try to mitigate things.
As of right now, ani.social is slowly working its way through this activity backlog. As I write this, it is about 11 hours behind world (see here).
The root cause of kbin causing spam seems to be some kind of technical bug with kbin. The users causing the spam seem to be real humans rather than bots, but efforts to get the kbin dev’s attention to the issue have so far not been successful from what I can tell. The world admins earlier today were debating simply defederating from kbin entirely due to the problem.
I wonder if Mbin instances are also affected. The Mbin devs should be easier to reach.
We have a couple people in the lemmy support matrix and have been keeping a close eye on things. So far I have not witnessed or heard it happening for any Mbin instance. The code has diverged a lot at this point, so it’s hard for me to say whether it’s something that could affect us. But, obviously the consequences of this are being felt severely by instance owners, so like I said, we’re trying to watch out for anything, and keeping track of metrics / queues.
Thanks. Seems to have started only yesterday. I wonder if this is supposed to be some kind of April Fools by bringing down the network or something.
efforts to get the kbin dev’s attention to the issue have so far not been successful from what I can tell.
ernest has made several updates in the past couple of weeks.
I am just relaying what the world admins have been saying in matrix and here on kbin. I am hopeful that the issue can be resolved because losing kbin users would suck.
You’re good, was just providing more up to date information because I know not everyone would have seen it.
As you said, it’s mostly about working through the backlog at the moment, but hopefully they’ve got it under control. Ernest has also been getting some help more recently, and is recruiting for more, so hopefully any situation can be delt with more promptly in the future.
But it’s just one of those things that is going to happen on a platform that is run the way most of the fediverse is - by individuals on their own set ups, which I think can take people who are used to corporate platforms (99% of us) some getting used to. It just takes a little more patience, but the work is being done.
Like others have said, .world federation seems to have some troubles ATM in general.
Notices it with my own posts in the sense that the vote counts appear higher on world because their votes are taking a while to be federated out to other instances.
There’s some talk about migrating active /c’s off lemmy.world because it’s overloaded with active communities.
The sheer load is causing federation of posts and comments to time out because .world can’t respond fast enough.
This has been especially noticed with instances that are far away geographically from .world, where internet lag time plus overloading leads to many more timeouts.
