1079
you are viewing a single comment's thread
view the rest of the comments
[-] grabyourmotherskeys@lemmy.world 330 points 11 months ago

I haven't read the article because documentation is overhead but I'm guessing the real reason is because the guy who kept saying they needed to add more storage was repeatedly told to calm down and stop overreacting.

[-] krellor@kbin.social 169 points 11 months ago

I used to do some freelance work years ago and I had a number of customers who operated assembly lines. I specialized in emergency database restoration, and the assembly line folks were my favorite customers. They know how much it costs them for every hour of downtime, and never balked at my rates and minimums.

The majority of the time the outages were due to failure to follow basic maintenance, and log files eating up storage space was a common culprit.

So yes, I wouldn't be surprised at all if the problem was something called out by the local IT, but were overruled for one reason or another.

[-] otl 59 points 11 months ago

and log files eating up storage space was a common culprit.

Another classic symptom of poorly maintained software. Constant announcements of trivial nonsense, like [INFO]: Sum(1, 1) - got result 2! filling up disks.

I don't know if the systems you're talking about are like this, but it wouldn't surprise me!

[-] DukeMcAwesome@lemmy.dbzer0.com 38 points 11 months ago

You gotta forward that to Spunk so your logs ain't filling up the server generating them. Plus you can set up automated alerts for when the result stops being 2.

This message brought to you by Big Splunk.

[-] wolo@lemmy.blahaj.zone 25 points 11 months ago

I think you missed a letter...

[-] Blooper@lemmy.world 20 points 11 months ago

I always make sure my logs are covered by Spunk.

[-] PhatInferno@midwest.social 1 points 11 months ago

spunking my logs is one of my favorite pass times

[-] Dope@lemmy.world 3 points 11 months ago* (last edited 11 months ago)

~~Big Splunk~~ Missed letter? You mean Big Spunk, right?

[-] SzethFriendOfNimi@lemmy.world 23 points 11 months ago

And yet that’s probably there because sometime, somewhere, it returned 1.9 or 2.00001 or some such nonsense.

[-] Klear@sh.itjust.works 5 points 11 months ago

1 + 1 = 2.000001 for sufficiently large (but not by much) values of 1.

[-] Pat12@lemmy.world 4 points 11 months ago

this is software speciifcally for assembly line management?

[-] Anomalous_Llama@lemmy.world 16 points 11 months ago

There is specific software for everything

[-] afraid_of_zombies@lemmy.world 0 points 11 months ago

Yeah a few levels.

Level 1: complex stand alone devices, mostly firmware.

Level 1a. Stuff slightly more complicated than a list of settings, usually for something like a VFD or a stepper motor controllers. Not as common.

Level 2 PLCs, HMIs, and the black magic robotic stuff. Stand alone equipment. Like imagine a machine that can take something, heat it up, and give it to the next machine.

Level 3: DCS and SCADA. Data control center and whatever SCADA stands for, I always forget. This is typically for integrating or at least data collection of multiple stand alone equipment for level 2.

Level 4: the integration layer between Level 3 and whatever means the company has for entering in sales.

Like everything in software this is all general. Some places will mix layers, subtract layers, add them. I would complain about the inconsistent nature of it all but without it I would be unemployed.

[-] Pat12@lemmy.world 1 points 11 months ago

Level 1a. Stuff slightly more complicated than a list of settings, usually for something like a VFD or a stepper motor controllers. Not as common.

Level 2 PLCs, HMIs, and the black magic robotic stuff. Stand alone equipment. Like imagine a machine that can take something, heat it up, and give it to the next machine.

Level 3: DCS and SCADA. Data control center and whatever SCADA stands for, I always forget. This is typically for integrating or at least data collection of multiple stand alone equipment for level 2.

Level 4: the integration layer between Level 3 and whatever means the company has for entering in sales.

Like everything in software this is all general. Some places will mix layers, subtract layers, add them. I would complain about the inconsistent nature of it all but without it I would be unemployed

Is this specific software engineering languages? or is this electrical engineering or what kind of work is this?

[-] afraid_of_zombies@lemmy.world -1 points 11 months ago

I am having problems understanding your questions. I generally operate on level 2 and we typically use graphics based languages when we implement scripting languages to do graphical languages. The two most common graphic languages are FBDs and Ladder-Logic. Both have a general form and vendor specific quirks.

For scripting I tend towards Perl or Python, but I have seen other guys use different methods.

Level 3 use pretty much the same tools. Level 4 I have in the passed used a modbus/tcp method but this isn't something I can really say is typical. One guy I know used the python API to do it.

[-] Pat12@lemmy.world 1 points 11 months ago

oh, thank you

my background is not in engineering which explains my confusing questions

[-] DontMakeMoreBabies@kbin.social 76 points 11 months ago

I'm this person in my organization. I sent an email up the chain warning folks we were going to eventually run out of space about 2 years ago.

Guess what just recently happened?

ShockedPikachuFace.gif

[-] TheBat@lemmy.world 25 points 11 months ago

You got approval for new SSDs because the manglement recognised threat identified by you as critical?

Right?

[-] vagrantprodigy@lemmy.whynotdrs.org 19 points 11 months ago

Literally sent that email this morning. It's not that we don't have the space, it's that I can't get a maintenance window to migrate the data to the new storage platform.

[-] IMongoose@lemmy.world 9 points 11 months ago

Sometimes that person is very silly though. We had a vendor call us saying we needed to clear our logs ASAP!!!!! due to their size. The log file was no joke, 20 years old. At the current rate, our disk would be full in another 20 years. We cleared it but like, calm down dude.

[-] mdd@lemm.ee 9 points 11 months ago

Can't you just add a few external USB drives? (heard this more than once at an NGO think tank.)

[-] grabyourmotherskeys@lemmy.world 12 points 11 months ago

I mean I've worked at a hosting company that had a bunch of static sites running off an SSD connected by usb to the server so this did happen back in the day. I try not to think about those days.

"What's that? Your accounting front end that's built in obsolete front page code on an Access database isn't working again? It's probably a file lock, I'll restart IIS."

[-] dojan@lemmy.world 27 points 11 months ago

Ballast!

Just plonk a large file in the storage, make it relative to however much is normally used in the span of a work week or so. Then when shit hits the fan, delete the ballast and you'll suddenly have bought a week to "find" and implement a solution. You'll be hailed as a hero, rather than be the annoying doomer that just bothers people about technical stuff that's irrelevant to the here and now.

[-] lemmyvore@feddit.nl 16 points 11 months ago

Or you could be fired because technically you're the one that caused the outage.

[-] dojan@lemmy.world 10 points 11 months ago

Damned if you do, damned if you don't!

[-] Awkwardparticle@artemis.camp 7 points 11 months ago

The ultimate goal is having no downtime. Ballast gives you that result. The cost of downtime far larger than wasting extra space for ballast.

[-] Malfeasant@lemm.ee 2 points 11 months ago

Except then they'll decide you fixed it, so nothing more needs to be done. I've seen this happen more than once.

[-] IWantToFuckSpez@kbin.social 11 points 11 months ago* (last edited 11 months ago)

And was fired for not doing his job which management prevented him from doing in the first place

this post was submitted on 06 Sep 2023
1079 points (99.4% liked)

Technology

57175 readers
3847 users here now

This is a most excellent place for technology news and articles.


Our Rules


  1. Follow the lemmy.world rules.
  2. Only tech related content.
  3. Be excellent to each another!
  4. Mod approved content bots can post up to 10 articles per day.
  5. Threads asking for personal tech support may be deleted.
  6. Politics threads may be removed.
  7. No memes allowed as posts, OK to post as comments.
  8. Only approved bots from the list below, to ask if your bot can be added please contact us.
  9. Check for duplicates before posting, duplicates may be removed

Approved Bots


founded 1 year ago
MODERATORS