I dont Think people properly understand they can be on any server. And join multiple communities. And it all Show up in their Feed. They don’t Need to worry about “which community has the Most Users”
It disregards the benefits of a distributed platform. Imagine if the admins went rouge, or the server data was irreversibly lost, suddenly all that content would be gone or under the authoritarian rule of the admins. Bit dramatic but you get the point.
If the majority of content is on there, we’ve quite literally taken a decentralised system and centralised it lol
On a more technical level, it takes quite some ressources for a server to broadcast their communities to all other lemmy instances.
“Receiving” a remote community is just reading data and inserting it in your instance. But if a community is hosted on your instance, you have to send that data to each and every instances with at least one user subscribed to it.
So it’s really better for everyone to spread out on as many instances as possible. The only thing I would recommend before setting up a community (or your user account) on an instance is to check if you align with their moderation rules/code of conduct.
Communities are inherently tied to the instance on which they are created and cannot be moved. If the instance is overloaded then that community will not federate properly. If the instance goes down nobody can post to the community. If the instance goes away that community goes away (except for the “cache” that other instances have).
Hmm. I’m not sure if that’s the case.
I’m interested to see what the plan is for account migration. Weather posts will follow the user. Or stay with the instance.
Migration of ActivityPub stuff is pretty rough… Everything has an ID, and that ID is the URL, so the ID of the post you replied to is literally https://lemmy.nrd.li/comment/227095… AFAIK there are some (non-standard, at least not in core AP) ways you can mark things to be like “yeah, this moved to over here”, but that isn’t built in to the spec so whether those mechanisms actually work is a crapshoot.
I’m talking purely from an ActivityPub/Activity Streams/Activity Vocabulary/JSON-LD perspective. There are some other local identifiers for things in Lemmy, but those do not matter for the purposes of federation. Any Object that is federated is expected to have an ID that is a URL at which you can make a GET request with the proper Accept header and you will get the latest version of that Object. AFAIK there is no provision for IDs to change.
I think we should expect/aim to just have some “mass repost script” that can take an extract of a community’s content and just “repost” it on a new community.
Basically, a script that would “replay” a community in one go. I don’t know if you could create “new comments” that immitates perfectly the original commenter but that would be the idea for a quick and very dirty “community mover script”.
A bit like in GIT when you want to change/remove a specific commit, you can only replay/rebuild everything from the start by creating new everything posts/comments.
Yes. Because there’s no centralised list of communities, searching is extremely difficult.
Or if not, very time consuming.
Following every iteration of every node.
The best way I can think of at the moment is a searchable website that gives you a link to click to seamlessly subscribe to them directly.
It’d be fine if the website is user submitted rather than having to interrogate all the servers on the back end, because the results would have seen a human eye and be better quality.
It doesn’t quite all show up the feed no matter what instance someone is on. In order for content to federate on an instance someone on that instance has to directly access it. I think this is why small niche instances appear to have a trickle of content on “all”.
One person on your instance needs to join for the community to appear. That is the major benefit of joining a large instance, you don’t need to search other instances or use 3rd party tools to find new communities.
Unless I am mistaken, when the instance you sign up with dies, so does your account? Obviously your content and potentially profile will exist in some state, but you would no longer be able to authenticate, so for all intents and purposes your account is gone.
While that won’t matter for some, for others that means there is some importance in the decision of where you create your account. Since, once that instance decides to shut down (or if it happens to defederate,) your account goes with it.
Which is exactly why you should self-host. No one to blame but yourself when your instance goes down/away.
Sadly this idea doesn’t mesh well with how communities work given those are inherently tied to an instance, unlike e.g. hashtags on Mastodon. It would suck if some community goes away just because the instance admin got tired of running it.
Yeh. But that’s happened with some of the biggest instances too. I know there are plans to be able to migrate your profile from one instance to another. Once that’s implemented, no reason to mass bombard any particular instance.
Haha, happened to my original account as well actually. lemme.ml must have had a rollback soon after I created my account last month, because it was gone when I tried to log in a week or so ago. So I get your point.
Out of curiosity what has the disk usage growth looked like so far for your lemmy instance? I occasionally selfhost but I’m not a hardcore datahorder or anything so the replication of data from instances you subscribe to has me on the fence.
Lady i checked, it was about 21g used from a 1tb ZFS pool.
My instance isn’t minuscule though. Few months old and only 20 users.
I’m curious about longer term growth though. No idea how long 1tb will last, but I have more of need be.
Damn that setup is no joke. 21GB in a few months initially sounded like a lot to me… but I decided to math it out.
Lets say the 20gb was across 1,2 or 3 months…
Time till 1tb would fill up.
+------+-----------+----------+----------+||3 months |2 months |1month||1 TB |~12 years |~8 years |~4 years |+------+-----------+----------+----------+
That data usage is looking pretty reasonable… Even 20gb per month is something that wouldn’t be too hard to keep up with and I’m sure eventually there’ll be a way to clean up old posts that no one on your instance saved or commented on if you are trying to save space. I’d start to worry if disk usage was hitting closer to 40gb a month.
i wish there was a way to show the growth over time, because obviously some is taken up by the OS, then all the initial setup of lemmy. I’ll keep an eye on it as it grows.
Grafana + something like influxdb+telegraf would do the trick. It sounds like you don’t have metric gathering like that on your instance? If that’s the case I’m surprised you don’t when you’re running with a full server rack haha.
I’m hosting one right now. Lemmyunchained.net
But in will have to Limit Users at some point.
I dont Think people properly understand they can be on any server. And join multiple communities. And it all Show up in their Feed. They don’t Need to worry about “which community has the Most Users”
Yes they can be on any instance, but I’m starting to get worried about the number of communities that are on Lemmy.world
Why is that worrying?
It disregards the benefits of a distributed platform. Imagine if the admins went rouge, or the server data was irreversibly lost, suddenly all that content would be gone or under the authoritarian rule of the admins. Bit dramatic but you get the point.
If the majority of content is on there, we’ve quite literally taken a decentralised system and centralised it lol
On a more technical level, it takes quite some ressources for a server to broadcast their communities to all other lemmy instances.
“Receiving” a remote community is just reading data and inserting it in your instance. But if a community is hosted on your instance, you have to send that data to each and every instances with at least one user subscribed to it.
So it’s really better for everyone to spread out on as many instances as possible. The only thing I would recommend before setting up a community (or your user account) on an instance is to check if you align with their moderation rules/code of conduct.
No. I think the other instances would need to purge that content right? I could be wrong.
Assuming it’s federated. And someone from your server is subscribed to that community.
Communities are inherently tied to the instance on which they are created and cannot be moved. If the instance is overloaded then that community will not federate properly. If the instance goes down nobody can post to the community. If the instance goes away that community goes away (except for the “cache” that other instances have).
Hmm. I’m not sure if that’s the case. I’m interested to see what the plan is for account migration. Weather posts will follow the user. Or stay with the instance.
Migration of ActivityPub stuff is pretty rough… Everything has an ID, and that ID is the URL, so the ID of the post you replied to is literally
https://lemmy.nrd.li/comment/227095… AFAIK there are some (non-standard, at least not in core AP) ways you can mark things to be like “yeah, this moved to over here”, but that isn’t built in to the spec so whether those mechanisms actually work is a crapshoot.Looking at my sql databases, I noticed there’s other identifiers on users and content. Not the url.
It may be that the url is linked to the ID. And that ID can just change.
I’m pretty much a noob. Just a lurker on the matrix chats.
I’m talking purely from an ActivityPub/Activity Streams/Activity Vocabulary/JSON-LD perspective. There are some other local identifiers for things in Lemmy, but those do not matter for the purposes of federation. Any Object that is federated is expected to have an ID that is a URL at which you can make a
GETrequest with the properAcceptheader and you will get the latest version of that Object. AFAIK there is no provision for IDs to change.Understood. I don’t know how the devs intend to overcome that challenge. If if it’s even a realistic goal. I just know it gets discussed often.
I think we should expect/aim to just have some “mass repost script” that can take an extract of a community’s content and just “repost” it on a new community.
Basically, a script that would “replay” a community in one go. I don’t know if you could create “new comments” that immitates perfectly the original commenter but that would be the idea for a quick and very dirty “community mover script”.
A bit like in GIT when you want to change/remove a specific commit, you can only replay/rebuild everything from the start by creating new everything posts/comments.
Or maybe that’s a terrible plan ;)
When lemmy.world will disappear, that’ll be a lot of communities (and valuable information) that go with it.
I don’t know if that’s the intention of Lemmy. It’s not Reddit. It’s not an encyclopaedia.
But I get that it would be annoying.
My understanding is that other instances would need to purge Lemmy.works for that info to all disappear.
Can you move a community once it’s created?
Nope, but that would be an awesome new feature!
Nope, it has to manually be setup again, and you’d have to get everyone to subscribe to the new one as well.
In practice right now it can be a bit schetchy tbh. Finding and subscribing to them is flakey and searching can be a bit hit and miss too.
When it does all work both smoothly and seemlessly then we’ll be golden.
Yes. Because there’s no centralised list of communities, searching is extremely difficult. Or if not, very time consuming. Following every iteration of every node.
I’m not sure how that can be overcome.
lemmyverse.net
The best way I can think of at the moment is a searchable website that gives you a link to click to seamlessly subscribe to them directly.
It’d be fine if the website is user submitted rather than having to interrogate all the servers on the back end, because the results would have seen a human eye and be better quality.
It doesn’t quite all show up the feed no matter what instance someone is on. In order for content to federate on an instance someone on that instance has to directly access it. I think this is why small niche instances appear to have a trickle of content on “all”.
There is a lemmy seed script you can use as an admin that gets you a “default sub” experience https://github.com/Fmstrat/lcs
Sorry. Was a typo. Says can join* any community and it show up on the feed. They need to join first. Yeas.
One person on your instance needs to join for the community to appear. That is the major benefit of joining a large instance, you don’t need to search other instances or use 3rd party tools to find new communities.
Unless I am mistaken, when the instance you sign up with dies, so does your account? Obviously your content and potentially profile will exist in some state, but you would no longer be able to authenticate, so for all intents and purposes your account is gone.
While that won’t matter for some, for others that means there is some importance in the decision of where you create your account. Since, once that instance decides to shut down (or if it happens to defederate,) your account goes with it.
Which is exactly why you should self-host. No one to blame but yourself when your instance goes down/away.
Sadly this idea doesn’t mesh well with how communities work given those are inherently tied to an instance, unlike e.g. hashtags on Mastodon. It would suck if some community goes away just because the instance admin got tired of running it.
Yeh. But that’s happened with some of the biggest instances too. I know there are plans to be able to migrate your profile from one instance to another. Once that’s implemented, no reason to mass bombard any particular instance.
Haha, happened to my original account as well actually. lemme.ml must have had a rollback soon after I created my account last month, because it was gone when I tried to log in a week or so ago. So I get your point.
Out of curiosity what has the disk usage growth looked like so far for your lemmy instance? I occasionally selfhost but I’m not a hardcore datahorder or anything so the replication of data from instances you subscribe to has me on the fence.
Lady i checked, it was about 21g used from a 1tb ZFS pool.
My instance isn’t minuscule though. Few months old and only 20 users. I’m curious about longer term growth though. No idea how long 1tb will last, but I have more of need be.
(This is my little lab)
Damn that setup is no joke. 21GB in a few months initially sounded like a lot to me… but I decided to math it out. Lets say the 20gb was across 1,2 or 3 months…
Time till 1tb would fill up. +------+-----------+----------+----------+ | | 3 months | 2 months | 1 month | | 1 TB | ~12 years | ~8 years | ~4 years | +------+-----------+----------+----------+That data usage is looking pretty reasonable… Even 20gb per month is something that wouldn’t be too hard to keep up with and I’m sure eventually there’ll be a way to clean up old posts that no one on your instance saved or commented on if you are trying to save space. I’d start to worry if disk usage was hitting closer to 40gb a month.
i wish there was a way to show the growth over time, because obviously some is taken up by the OS, then all the initial setup of lemmy. I’ll keep an eye on it as it grows.
Grafana + something like influxdb+telegraf would do the trick. It sounds like you don’t have metric gathering like that on your instance? If that’s the case I’m surprised you don’t when you’re running with a full server rack haha.
I always just use the proxmox data. I’ll check out grafana.
I’ve seen something like 8 comments pointing people towards their own servers.
Which essentially guarantees a level of community fragmentation as to prevent community growth, cohesive, or general activity does it not?
Ideally each community “group” would have their own Lemmy instance.
Yeh. There’s a few servers that donut really well. Lemmy.nsfw is probably the best example I think.