A community for everything relating to the GNU/Linux operating system (except the memes!)
Also, check out:
Original icon base courtesy of lewing@isc.tamu.edu and The GIMP
am regularly amazed that we pretend folders are the right way to organise files. They’re entirely arbitrary. Every competent file system ignores them to its best ability. Why can’t I have a file in two folders? Why does one have to be a “reference”? Why can’t I filter for files that exist in 3 folders with X extension?
We’ve been played for absolute fools.
you can do exactly what you're talking about, you just didnt bother to look it up. it is done the way it is because it is intuitive to organize things that way and it is a good way to make the actual organization of data on a storage device human readable. most people have no use for organizing things the way you are talking about 90% of the time, bit because usecases do exist where it is helpful it has always been possible to do that on linux. depending on the apps you use it may not be exposed graphically, but it can be done.
You can have a file in 2 folders, they're called hardlinks
Why can’t I have a file in two folders? Why does one have to be a “reference”?
You can do this... Hard links. Neither file is more "canonical" than the other.
At my company we use M-Files, which is a document storage system that prides itself in not using folders. “No more searching for the file in thousands of folders”, they proclaim. It’s all a huge dump of files. To find files you need to tag them when checking them in. Later you search via these tags.
Guess what happens: All documents are either untagged or they’re tagged with wildly unhelpful tags. So in reality you can’t find shit. You can’t even make a sensible guess as to where a file might be and check the 3–5 folders that come to mind, because there are no folders.
M-Files is a black hole for information. No, scratch that. Even black holes radiate out the information they receive. M-Files doesn’t.
sounds like the SharePoint one of my previous employers used. Now, SharePoint supports folders! but, using it through Teams, like everyone did, with tens of thousands of files haphazardly vomited onto it randomly, meant that Teams literally can't load the file list fast enough. So, again all information goes there to die.
It was not nice.
No, scratch that. Even black holes radiate out the information they receive. M-Files doesn’t.
That is an amazing zinger!
In my last company we used a system called windchill. Technically they had folders. Previously we used a different system. But when we switched to windchill no one had time to actually sort and organize the tens of thousands of documents. As a result everything just got dropped in the root folder.
To make it worse there was no enforced naming scheme... Plan for... Thing's plan.... Protocol for execution of thing... Ip of thing.... Thing's up... Protocol of thing.... Plan of thing... All valid. And in 5 years when your 3rd replacement is trying to find it... Alcoholism is a serious disorder
Why can’t I have a file in two folders? Why does one have to be a “reference”?
All files are references. But you have always been able to put a reference to one file in multiple folders by using hard links.
Why can’t I filter for files that exist in 3 folders with X extension?
find dir1 dir2 dir3 -name '*.x' -type f
We have hard links, but is there any good UI out there for them? I only know of using the ln command directly. Or put another way, do you know of anyone who actually uses hard links in a way similar to how a tagging filesystem would be used? What are the obstacles that prevent this use case from being easy or discoverable enough to be in common use?
With a tagging system you can remove tags without fear of losing file data. But with hard links you could easily delete the last link without realizing that it's the last link, and then the file is gone.
That relates to another issue: in a tagging system you can look at file metadata to see all of the file's tags. Is there a convenient way to do that with hard links? I see there is find . -samefile /path/to/one/link, but requiring a filesystem scan is not optimal.
No, in nearly every case, you never want a hard link. You want one file, and symlinks to it. (Technically every file is a hard link to an inode, and subsequent ones are just additional links to the same inode.) In ext4, you can't easily get a list of links to an inode, you have to scan the filesystem and look for duplicates. Other filesystems might make this easier.
You shouldn't try to use a tree filesystem to approximate a tagged database. Use the appropriate tool for the job.
That's not an unreasonable answer. But I find this thread a little frustrating. As I see it, it's gone like this:
Why bring up hard links if people shouldn't use them for the requested use case? I mean, I do think your original reply was interesting and relevant as a starting point to get to what I think OP has in mind. But that line of thinking does require getting into how to use hard links for a non-hierarchical workflow.
I feel like OP was trying to start a discussion about what might be, if things were different. I tried to reply in the same spirit. I feel like I'm asking, "What if things were different?", and I'm being told "It doesn't work that way." Which doesn't feel like an especially helpful response to me.
If my grandmother had wheels, she'd be a bicycle. Don't try to force a hierarchical filesystem into other applications. Asking "what it if was different" doesn't make sense, because if ext4 was a tagging filesystem, it would be tagfs! Tagfs and others already exist, you can use them today! Go crazy, use object or document stores! Embed everything in xml! But for operating systems or actual human interface, those are terrible ideas, which is why we don't do that.
I brought up hard links to say "yes, you can do that, but it doesn't mean you should". There are few cases for hard links. The only one I've seen in actual use is in media downloaders, where a file gets downloaded to one folder, then hardlinked from a library folder, so that you don't have two copies of that video. The library (jellyfin) can do whatever it wants to the link (move, rename) while the downloader (qbittorrent) can still keepit for seeding, and either side can happily delete its copy without affecting the other.
Most people don't need a file to be in two places at once, it's more confusing than convenient. And if they do want two of a file at all, they almost certainly want them to be separate copies so that the original stays unmodified when they edit the second one. Anyone who really wants a hard link is probably comfortable with the command line, or should get comfortable.
The Mac actually kind of gets the best of both worlds, APFS can clone a file such that they aren't hard links but still share the same blocks of data on disk, so the second file takes up no more space, and it's only when a block gets edited that it diverges from the other one and takes up more space, while the unmodified blocks remain shared. It happens when copy-pasting or duplicating a file in the Finder as well as with cp on the command line. I'm sure other modern file systems have this as well.
Yep. Even NT was doing it for decades. Though it was pretty discouraged on the Microsoft side.
Is this rage bait?
No no, you see, most people like organisation. Also, it's intuitive as it is analagous to actual folders that store actual files. It would be kind of weird if you could store a file in multiple folders. What would that even mean? If you delete a file from one folder, is it deleted in the other? Folders aren't meant to be labels (labels are labels), they're locations. Your toothbrush is in the bathroom, pots and pans in the kitchen, etc.
Skill issue.
I know, instead of folders, we could use "shelves" and the Dewey Decimal System.
Tree-like hierarchy is used all over the place, including computers, because it's a useful and easily understood way to organize information.
Why can’t I have a file in two folders?
You can. man ln
Why does one have to be a “reference”?
I don't know what you mean by that. If you mean a link target, it doesn't. A file is canonically identified by its inode (or equivalent), not where it appears in a directory tree.
Why can’t I filter for files that exist in 3 folders with X extension?
You can. Common tools like find can do this, as can some file managers like Dolphin, and various indexing tools.
If you mean to ask why that sort of indexing/filtering isn't built in to most filesystems, consider compatibility: Practically no software exists that would know how to take advantage of it. Also consider what it would mean for a filesystem to filter by files that exist in 3 folders if that filesystem doesn't use folders. :)
(BTW, that "extension" concept doesn't exist in most modern filesystems. Any .xyz suffix you see in the ones that don't come from Microsoft is just part of the file name, with no special meaning. Some programs try to guess at content type based on common file name suffixes, but that is unreliable and has nothing to do with the fs.)
Since you're interested in this topic, though, maybe have a look at different approaches to data storage that have been tried over the years. To get you started:
https://en.wikipedia.org/wiki/Object_storage
https://en.wikipedia.org/wiki/File_system#Database_file_systems
Not sure why people defend an archaic organization form here - reflex ?
You are perfectly right that files and folders are simplistic, and should naturally adapt to the pov that are more information rich/valuable. Hoomans tend to collapse a high-dimensional structures to 0D to 3D, so we can manage the information. In that sense, a std hierarchy is only ONE pov over a ton of pov over the same content. A standard hierarchy is only a low 2D dimension structure that are our first attempt at organizing information. It's not wrong - just imprecise af.
Anyway, hardlinks are a small step up, can build wild static structures (like a oneshot filesystem in Guix), but is cumbersome to control in multi-dimensional information structures. Likely not what you want, but look into fuse file systems if you want to move on to a dynamic file system hierarchy. An interesting one is a tag file system. It turns a standard limited hierarchy into a much more dynamic file-structure where a file can - and does - belong to a bunch of tags - file type, size, group, comments, whatnot. There are many many fuse fs that can convert anything into a better structured file system. Tagging is a step up from a dumb 2D hierarchy, but maybe a graph file system is the ultimate freeform dynamic filesystem that can present all the pov's we could possible need ?
Go for it.
Wait, I've got it: what if we put every file in one folder, and then hard-link the files we need higgildy-piggildy all over the place!
We can call it WinSxS!
I can't say I disagree, but I am ignorant of what the alternatives would be. A tagged database of files so you can query by tag, filename, or such?
You have to think in 'information dimensionality'. A yes/no toggle is 0D, a list is 1D, a list of lists (std hierarchy) are 2D, a list of list of list are 3D etc. All information storage types are one of these dimensions. Think of a graph-base file system with nodes and edges between everything. Now, imagine a filesystem where you flick a switch and the whole structure shows another pov ? Maybe you want the whole thing to be shown as file-type hiearchy, or only parts of it. Maybe you need to show movement in the structure, so everything are in a temporal/spatial hierarchy, maybe you are only interested in dependencies ? Relations ? Other 'weird' metrics ? ..and so on. The main problem is to manage, find and show the needed information in a higher-dimentional fs.
Technically a normal file is also a list, or another ordered structure, but in this sense, they are just a node with further dimensionality.
There's a TON of information layers locked away in our normal filesystem hierarchy, so OP are perfectly right, and people here have no imagination or even a world model of information structures..
A hierarchical tag structure would let you mimic a folder structure, but free you from many of its constraints
What is your alternative non-folder solution?
SQlite 😅
Can you please save this somewhere you'll find it in 10 years?