30
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 23 Sep 2024
30 points (94.1% liked)
Linux
48735 readers
1011 users here now
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Rules
- Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
- No misinformation
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
founded 5 years ago
MODERATORS
I am trying to transfer data via USB at high speed without data corruption, silent resets and occasional device disconnects. Those are things that happen because the USB controllers on my motherboard made by AMD with some help from ASMedia do not function correctly at the speed they advertise. So given the problem the right solution is to get a firmware or hardware fix for these USB controllers, however that's unlikely to happen. So I'm trying to find a workaround. I already have one (PCIe add-in card) but now I'm also testing running the bad controllers at half-speed which seems successful so far but I was wondering if there's a way to do it in software. I'm currently bottlenecking the links by using 5Gb hubs between the controllers and the devices.
A bit of searching brought me here - would this suit your purposes?
Edit: amusingly, one of the replies to the original question also points out that this is yet another classical example of the X-Y problem
Unfortunately it won't because the transfers are happening between ZFS and the hardware storing the data so I can't control the data rate at the application level (there are many different applications) or even at the ZFS level. This is why in this particular case I'm stuck with a potential hardware-related workaround. I mean I could do something stupid like configuring a suboptimal recordsize in ZFS but there could still be spikes and I'd prefer to get the hardware to stop losing bits and hoping ZFS would catch that. Decreasing data rates is a generally acceptable strategy to deal with signalling issues, if the decreased rate is usable for the application at hand. In my case it is.
Heh, we’re still on the X-Y problem to a degree.
I’d recommend another top-level reply to your post, or a new post, describing precisely how your hardware and ZFS pools are set up, alongside a description of the firmware/stability issues you’re seeing, and solutions you’ve tried already. We’re a bit far down a comment chain at the moment, so you’ll probably get more engagement that way. Not trying to be an ass - this actually sounds like an interesting (and, I’m sure, obnoxious) problem. Putting all the cards on the table will help people give you a more complete answer more quickly.
The thing is that I'm already at the last couple of leaves in the investigation tree and I'm not willing to change anything upwards of the USB driver level. That's why there isn't much point in getting people to spin their wheels for solutions I can't or won't apply. If I was completely unable to get the data corruption and disconnects under control, I'd trash the system and replace it with Intel. Fortunately, a PCIe add-in USB controller seems to work well so I avoided the most costly solution. At this point I don't actually need to get the motherboard ports to work well but I'm curious to follow down the signalling rabbit hole because I'm not the only one who's having this problem and the problem doesn't affect just this one use case. If I find a solution like an in-line 5Gb USB hub (reduces data rate), or just using USB-C ports instead of USB-A (reduces noise), or using this kind of cable instead of that kind, I could throw that as a cheaper workaround in this ZFS thread and elsewhere. The PCIe cards work but aren't cheap.
That all makes sense - I guess at this point, I’m simply trying to offer some constructive criticism about how to present nuanced problems that involve both hardware and software gremlins in a way that you’ll get the most productive conversation and interaction from the user base here.