Technology

42853 readers

85 users here now

This is the official technology community of Lemmy.ml for all news related to creation and use of technology, and to facilitate civil, meaningful discussion around it.

Ask in DM before posting product reviews or ads. All such posts otherwise are subject to removal.

Rules:

1: All Lemmy rules apply

2: Do not post low effort posts

3: NEVER post naziped*gore stuff

4: Always post article URLs or their archived version URLs as sources, NOT screenshots. Help the blind users.

5: personal rants of Big Tech CEOs like Elon Musk are unwelcome (does not include posts about their companies affecting wide range of people)

6: no advertisement posts unless verified as legitimate and non-exploitative/non-consumerist

7: crypto related posts, unless essential, are disallowed

founded 7 years ago

MODERATORS

MinutePhrase@lemmy.ml

How to deduplicate sentences in large .odt and .txt files? (lemmy.ml)

submitted 1 day ago by zdhzm2pgp@lemmy.ml to c/technology@lemmy.ml

11 comments fedilink hide all child comments

Over the years, I've downloaded a lot of old emails to my laptop, which I saved as .eml files and then converted to .docx, .odt, and .txt files (mostly the last two).

The unfortunate habit of almost everyone (including myself, but no longer!) of quoting the original message as part of an email exchange has left me with text file(s) full of repeated sentences and paragraphs. What I've tended to do is to dump all the text from a one on one correspondence into a single file ("Dad-Erinaceus Complete Correspondence.odt," for example) and then try cleaning it up and re-ordering the messages by date.

Apart from the emails, I have I guess what you'd call a "journal" which is a very long .odt file that runs to about 300 pages or so. Much of this has the same sentences and paragraphs over and over again, but sometimes with slight variations that I would like to keep. So far, in either .odt or .txt files, I've started by searching for the first sentence, deleting subsequent appearances of it, and then going on to the next sentence, and so forth. Very time consuming! Is there a faster (and safe) way to do this?

There is quite possibly a very simple solution to this that I haven't thought of, but I'd be much obliged for any suggestions.

you are viewing a single comment's thread
view the rest of the comments

[–] Zarobi@aussie.zone 2 points 23 hours ago* (last edited 22 hours ago) (1 children)

Inbox zero doesn't mean deleting all your emails lol. It just means getting them out of your inbox. I do this myself. For each email in my inbox I "do something" with it instead of just leaving it there. Put it in a folder, tag it, spam, delete, archive, anything. Personally I use 3 zones:

inbox: currently to-do. Things I need to reply to or action I "flag" so they're at the top. Everything else is moved either to
archive: things I might need in the future. Any important correspondence. Bills. Etc. The important part is it is "out of sight". I use labels to tag everything, I find this is easier to search and navigate. Plus if you put a label on the email, you know why you kept it, it forces you to quantify that reason.
trash. Auto deleted after 30 days in case I change my mind and actually need it. Most emails go to trash. Aggressively unsubscribe from newsletters.

[–] zdhzm2pgp@lemmy.ml 1 points 22 hours ago

🙂👍