75

cross-posted from: https://sh.itjust.works/post/63615439

The reality of how much damage AI mania is doing to our ability to run institutions effectively.

you are viewing a single comment's thread
view the rest of the comments
[-] HaraldvonBlauzahn@feddit.org 17 points 4 days ago* (last edited 4 days ago)

What is puzzling to me is there seem to be no companies which take a deliberate, measured, experimental approach.

No "You can use that but you have to report on difficulties or disadvantages".

No "you can use it but not when correctness is critical, errors could harm people, or it can directly affect things critical for our business".

No "Give us first an estimate in how long it will take, and then we will roll a dice, decide if it will be done using AI, and after 50 tries we will compare whether it makes a difference".

No "You can use AI, but you are responsible that the result works and is correct, no slack on this, and failure to do so gives you minus points".

No "You can use it, your max token budget is 100 USD per month".

No "You can use it, but do not input any company secrets, confidential data, medical records, or things like passwords and security-relevant data".

There seems to be zero risk management.

Imagine pesticides, radioactive substances, or highly toxic chemicals would have been introduced in such a manner.

See also this book chapter by Baldur Bjarnason.

[-] ArmoredThirteen@lemmy.zip 13 points 4 days ago

Companies doing that aren't making headline worthy mistakes

[-] cschreib@programming.dev 10 points 4 days ago

Our company develops software for medical devices. Risk assessment is everything, and we currently do "You can use AI, but you are responsible that the result works and is correct" and "You can use it, but do not input any company secrets, confidential data, medical records, or things like passwords and security-relevant data".

I guess the difference is we're too busy making things and we don't have the time to brag about AI on social media? Also we're small and don't have the reach anyway.

[-] atzanteol@sh.itjust.works 2 points 3 days ago

And it's not "news" in the same way that "I drove to work today without incident" doesn't make headlines.

Imagine pesticides, radioactive substances, or highly toxic chemicals would have been introduced in such a manner.

Sadly, there is no need to imagine. There is no limit to people's greed. And guardrails are only installed after there is no way to hide damage done.

[-] atzanteol@sh.itjust.works 1 points 3 days ago* (last edited 3 days ago)

Why don't you think it's not happening? Do you think headlines are the best way to determine what's happening in the world?

[-] HaraldvonBlauzahn@feddit.org 1 points 3 days ago* (last edited 3 days ago)

Well, show me any job ads that tell this.

Or is this somehow hidden in some kind of cat whistle?

[-] mspencer712@programming.dev 2 points 4 days ago

My employer does the first, second, fourth, and sixth of those. (And I’m in the uncomfortable position of simultaneously building internal tools to make our AI use more effective, and telling everyone who will listen that we’re just walking blindly into yet another Oracle or Broadcom situation.)

[-] jsnfwlr@lemmy.ml 1 points 4 days ago

No "You can use that but you have to report on difficulties or disadvantages".

No "You can use AI, but you are responsible that the result works and is correct, no slack on this, and failure to do so gives you minus points".

No "You can use it, your max token budget is 100 USD per month".

No "You can use it, but do not input any company secrets, confidential data, medical records, or things like passwords and security-relevant data".

These are all policies at the company I work for. Only its $100 AUD, not USD.

this post was submitted on 19 Jul 2026
75 points (100.0% liked)

Programming

27808 readers
379 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 3 years ago
MODERATORS