I remember an experiment in the late 90s where they had “AI” from then (fuzzy logic and some other tech) design EPROM chips that had to generate a very specific frequency, or something like that.
They ended up with 20 chips, and 20 different designs. They all worked, but…
Each one was programmed in a way that worked on that chip, and that chip alone. Copy the programming from one chip to another, and it would not work
Some chips had redundant circuits that connected to nothing, just sitting there. When those circuits were removed, the chip would fail too, even though those circuits in principe didn’t seem to do anything as, again, they weren’t connected to anything
Not a single one had an actual sense making correct solution
Basically the system just kept tweaking each chip until it guy something that worked
Current LLM’s are more powerful but still operate in many similar ways. You can’t ever trust them output so you’ll have to check everything it does manually to get any remote kind of trust in the system, but how do you even go to test whatever random crap comes out on an AI chip?
Government abducts people out of their homes