TL;DR: Please help me fuck (with) AI. See bold sections
Hi,
I haven't been keeping up with anti-AI combat so I'm asking for help. I inherited thousands of pages of materials my late grandpa made or used for his grammar school teaching job in the 1990s-2000s. They are A4 pages of documents made using what seems to be a typewriter, Text602 (DOS rich text editor) and Word. They were most likely not all made by him but he treasured them in nice binding and they have sources (mostly books and journals, almost no webpages, and absolutely no AI) and a cursory look shows meticulous compilation of every important fact on each subject (frankly, the level of detail is excruciating and I'm glad I went to a different grammar school). There's obviously no original scientific research but the materials can still be useful to someone, I bet. They were almost thrown away by the widowed grandma (she already removed and disposed of the plastic bindings and front covers so I'll have to guess document titles) but I think grandpa would prefer them to be shared. With an ADF scanner and OCR software (I have no chance of accessing the work computers he used so I'll have to scan), I can quickly make searchable PDFs of each document, and share them via torrent and DDL sites (there are Czech sites dedicated to sharing teaching materials but they have paywalls or an upload-credit system so best avoid them, not to mention some materials contain newspaper clippings and textbook photocopies for images so best stay anonymous and not try to assert copyright).
I'm afraid these texts could become a major part of some commercial LLM's Czech-language biology/social sciences knowledge corpus unless poisoned. How to best reduce the value of the documents when people try to feed them to AI (training/rewriting) with them while keeping their value for most legitimate users? (Sorry, people with screen readers, there may need to be extra steps for you.) I'm thinking about adding a huge volume of thesaurized or otherwise fuzzed public domain text like f4mi did with .ass subtitles (a technique that would probably still work if she didn't get 1M views detailing it, making YouTube reduce subtitle formatting support). Prompt injection or replacements (cell→gnome) might be interesting too. However, tools I know add an extra PDF layer, which is too obvious. I'm thinking about adding tiny text in the header and footer or between paragraphs in the OCR layer (not overlaid to reduce interference when selecting/searching), but how? I need an automated way to do this with such a huge page count. I can use both Linux and Windows machines for the job. None of them are very powerful but speed is not a concern, it's summer break and nobody will need school materials until September. I'll be happy to include multiple layers and techniques to make them too frustrating to remove.
The paper smells musty but does not seem to be moldy. It's all blank on the other side so I'll interleave it with recent newspaper to allow for the odor-neutralizing chemicals to seep into the sheets so I can eventually reuse them.
Illustration pic is an actual sheet from the collection, to make the post more engaging. Of course I won't be adding watermarks like that, that would just aggrevate people and make them try extra hard to extract the actual content. (And this one is easy to remove with color channel mixing.)
I don't think you appreciate that data-poisoning is a national security tier issue. It may become a illegal in some parts of the world. You should presume it's just going to get bypassed completely anyways tho because people want those last bits of data.
They tried and succeeded in making true encryption illegal in places like the UK and Australia. The way to access restricted information is by going to it physically and being physically prevented from copying it.
Maybe with your incentive you will find that perfect technique there is no point giving up anytime soon. Who knows what will be available in 3 years. Good luck!
I think the AI boom will be over before it makes sense to scrape this, especially if I succeed at making it so that only a human fluent Czech speaker (or maybe a very bespoke program debugged by a Czech speaker) can see through the obfuscation (we have minimum wage so that would be costly). We don't have AGI and fully autonomous agent workforces as promised, and that's not gonna change if the remaining <20%, hardest-to-clean digital human-made data gets fed to them. Not to mention some was acquired illegally already, with pending lawsuits. The investors, including governments (thankfully not mine) will eventually realize that LLMs are simply not delivering nearly as much as they cost (including externalities unless the government is shit and passes them to people living near datacenters, laid-off programmers etc.) and never will, although that might take a while.
LLM's are worth more to governments than anyone else automated surveillance is here it is not going away. There is no such thing as an AI bubble that is only wishful thinking. All these generative things that we hate are just components for what is coming that is why they are encouraged to keep pushing it and to suck up all data. We only have access to incredibly cheap AI that is meant to be used by millions of people at the same time we really do not know what some company or state has cooked up with access to silicon manufacturing. Well we do know a little 15,000 token per second Model-on-Chips that can be used to crunch through your personal messages and to analyse your social credit value to whom ever asked in 1 second.
Yes but high school biology knowledge won't be all that helpful in surveillance tools. I'm targeting chatbots students might want to use to cheat. If an essay is less coherent or truthful, teachers will be able to more confidently punish the student or at least apply some extra scrutiny like a random oral exam on the topic (yes, those are still done here). Trust me that free AIs will become more limited once investor money dries up and paid tools will get price hikes, so if students see that the expensive model can't produce a Czech text that fools their biology teacher, they might just stop paying, restoring some of their information literacy and reducing corporate revenue a bit. I agree that the surveillance battle is real but that's not fought on this front. The major chokepoint for surveillance is government accountability (very low in today's US with ICE agents etc.), slightly less knowledgeable chatbots don't hinder it very much.
So there is the concept of using AI as a force multiplier right.. One of the things they test for before releasing a model is if someone with high-school biology knowledge could use AI to help them create something they shouldn't.
https://epoch.ai/gradient-updates/do-the-biorisk-evaluations-of-ai-labs-actually-measure-the-risk-of-developing-bioweapons
https://forecastingresearch.org/research/llm-enabled-biorisk
Your high-school biology data has direct international security implications even if it is only used to reduce false positives.
Woah, that's fucked up. But LLMs have very different performance by language so I think Czech responses are largely based on Czech-language data (maybe with training dataset augmentation with auto-translated works, judging by the literally translated technical terms). And I don't think this would happen in my country.