Hell yeah, let’s hold them accountable for disinformation. They’ll be gone completely in a matter of months.
Want to get rid of that responsibility? Direct the user to the source. Oh wait, that’s just a search engine.
I think a better solution is to ban techbros from giving serious economic or cultural advice and take computers away from business majors.
Please don’t take them entirely away. Maybe just internet access? 30ish years had to do accounting by hand. In those green ledgers. It took approximately twelve times longer to do it by hand than to do it with a computer. And it made me shrimp like 5 times worse. I needed an architect’s table what angled the top of it in order to work properly but I could neither get one supplied by the employer nor afford to give one to the employer.
Not all technology is bad
Oddly specific gripe, I’ll allow it.
thank you i have others in jars in the back
I don’t get how some of these tech company CEOs who came up as engineers can be pushing this bullshit. I get once the company got big they started hiring business bros. But some big companies still have CEOs that were once engineers. You’d think they would know better.
What kind of engineer? Because while the physical world, with all of its mechanical and civil and aerospace engineers, has its shit figured out with professional standards and very clearly defined responsibilities and duties, the world of social engineers, tire engineers, procurement engineers, supply chain engineers, sandwich engineers, project engineers, lead engineers, and yes, software engineers, definitely is a little too loose with any definition for me to care that these ceos were once ‘engineers.’
Would be nice if regular legal and health advice was in any way affordable though
This reads as a way to protect white collar industries from the effects of AI without addressing the root problem–that AI does not actually think, and that it is little more than a meat grinder full of scraped data.
In other words, Artificial Stupidity. Why is it CALLED intelligent?
Why is it CALLED intelligent?
Because it is “intelligent” by definition. You’re conflating the word with “highly intelligent” or just “smart”.
Dogs are “intelligent” but can’t they write code, but we sometimes refer to dogs as “smart”.
A flatworm has intelligence but no one would call it smart.
it had that name for a really long time
a couple decades ago, a program learning was really impressive
I remember when LISP was available for my Atari 800.
Yes, I had the FULL 64K of memory installed.
marketing
I don’t see how you police/enforce this. The technology is out of the bag, people will find ways to access. Do we need age/location verification for this now too? What if I’m running a local agent? I don’t agree with this.
The law would allow you to sue whoever is running the chatbot. If you run your own LLM locally and take bad advice from it, then it’s your own fault.
So who gets sued. The guy who put the chat bot on the server and is running it or the chatbot software developer themselves?
Or both?
Walk me through how a company based and operating not in new york would be subject to any actions from this lawsuit.
I do agree it’s limited to a small scope of New York-based smaller LLMs, but if you read the news you know why exactly this bill occured - just now Mamdani gave up on a useless chatbot made with local budget by his predecessor Adams: https://www.thecity.nyc/2026/01/30/mamdani-unusable-ai-chatbot-budget/ It was indeed giving inaccurate legal recomendations on city’s website. I think the better result that can happen to that bill is it becoming a trend across cities and states as, I suspect, New York administration wasn’t the only one falling for this scam.
I may have become too cynical but, as is often the case when you dig deeper, this sounds like the result of lobbyists trying to protect licensing rather than people.
We can be dumb, but we’ve been doing web searches for legal and medical advice for ages because it is too damned expensive and time consuming to go to professionals for every little thing. Not to mention, doctors have so little time for you that it is hard to get them to listen to the whole story to make connections between symptoms.
The LLMs already tell you that they aren’t licensed professionals and, for many, provide citations for their sources (miles better than your typical health website).
As a personal anecdote, my son was having stomach pain but was planning to tough it out. He checked with ChatGPT and it recommended he go to the ER. He did, and if he hadn’t, he would likely be dead now. He spent 3 days in the hospital having his bowels unobstructed through a tube in his nose.
There is value in people having that kind of information at their fingertips.
Regulation is absolutely needed, but I would rather they focus on protecting us from AI being used for military purposes, mass surveillance, etc. rather than protecting citizens from ourselves.
Are you in the US? My take away here is American healthcare is bad but we’re treating the symptom not the disease.
Yeah, I’m in the US and I agree. Though it is going to take some serious change to treat the problem. In the meantime, this is at least a stopgap solution for people who don’t have a lot of options.
Wait, he thought he could sit that pain through at home? Your son is tough as nails. Give him a hug for me and everyone else who’s had that four day n-g tube delight.
Yeah, he is pretty tough. I wish I could hug him, he is about a 10 hour drive from me. That tube was nightmarish from what he’s told me.
if i were his parent, i would be giving him gentle reminders to drink more water. after teasing him for eating way too much corn or broccoli or whatever bastard fiber caused his obstruction (assuming he’s in a mental place he can handle the teasing)
He’s in his 20s so he is only slightly more likely to take my advice than he was as a teenager 😆
if he’s more likely to take the advice, tell him it came from someone who has had hundreds of bowel obstructions.
Just have them add a disclaimer or have the hosts be liable for what their chatbots say, stop adding bureaucracy just asking to get selective prosecuted and abused.
Name checks out.
Section 230 of the dmca is designed to allow platforms to exist because people can say whatever the fuck they want. But nobody should make a machine that says things they can’t control, and if you do you need to be disciplined for such irresponsibility.
You bring a regulation - can you really enforce it?
If implemented, that would just ban chatbots that use large language models. It’s not a terrible idea.
What would actually happen is that so-called AI chatbot systems would try to detect if someone is from New York and then try to exclude them from receiving medical or legal advice, fail, and then get sued and then pay a small fine, over and over again forever.
Sounds like a start. More is needed though.
The bill targets AI chatbots that impersonate licensed professionals — such as doctors and lawyers — and bars them from providing “substantive response, information, or advice” that would violate professional licensing laws or constitute the unauthorized practice of law.
It also mandates that chatbot owners provide “clear, conspicuous, and explicit” notice to users that they are interacting with an AI system, with the notice displayed in the same language as the chatbot and in a readable font size. However, the bill clarifies that this notice for users, which indicates that they are interacting with a non-human system, does not absolve the chatbot owners of liability.
AI in the legal field could be useful for assisting an actual legal professional in compiling precedent based against on-the-books laws, so long as it cites sources and they verify them.
In the medical field, it could be useful for spotting anomalies between multiple images such as X-rays or cross-referencing medical documents WHEN USED BY A PROFESSIONAL.
But the thing is, it should be a tool - carefully used - to enhance the existing profession, not replace actual professionals.
But the thing is, it should be a tool - carefully used - to enhance the existing profession, not replace actual professionals.
except in practice, the “professionals” just take the LLM’s word as unassailable and disengage their brains. funny that, the gap between theory and reality
Yup, but those are the cases that make the news. There’s always gonna be some stupid/lazy ones
tell me you haven’t worked with anyone in the medical industry without telling me you haven’t worked with anyone in the medical industry
source: 20 years as a medical accountant
I mean.
Is the wikipedia responsible for you reading an article about a law and then taking that as legal advice?
[Edit: if you are downvoting this, downvote away, but you owe an argument below as to why. I promise this exact argument will come up in the courts over this issue]
Wikipedia doesn’t give “legal advice”, it has information about these laws, with the sources cited.
That is very different than asking LLM anything and it throws you random bullshit from unknown sources, with no easy way to verify where it is from or if it is at all accurate.
Wikipedia doesn’t give “legal advice”, it has information about these laws, with the sources cited.
That is very different than asking LLM anything and it throws you random bullshit from unknown sources, with no easy way to verify where it is from or if it is at all accurate.
It seems like your argument is that because Wikipedia “gets it right” and has cited sources, it isn’t liable? Which I promise, is not how liability works.
What if it was Wikipedia versus “Some random sovcit facebook post” then? Is the Sovcit post liable because its sources are bullshit? Since there sources are random bullshit and or unknown, do they absorb liability? Again, its the same case, that is not how liability works.
People are going to have to acknowledge you can’t have it both ways.
Also…
with no easy way to verify where it is from or if it is at all accurate.
C’mon. Plenty of LLM’s can also hallucinate sources which are easily verified. And like with Wikipedia, one could go check them.
Wikipedia isn’t giving you advice, it’s giving you information. There is a big difference between me taking information and forming an opinion, versus being given an opinion by a system that is responding to a specific situation explained to it.
Also, people get in trouble for giving legal advice, artificial unintelligence('s companies) should as well.
Wikipedia isn’t giving you advice, it’s giving you information. There is a big difference between me taking information and forming an opinion, versus being given an opinion by a system that is responding to a specific situation explained to it.
Okay lets try this then:
Chat bots aren’t giving you advice, it’s giving you information. There is a big difference between me taking information and forming an opinion, versus being given an opinion by a system that is responding to a specific situation explained to it.
Show me the difference.
Also, people get in trouble for giving legal advice,
No, they don’t, unless they are genuinely misrepresenting their positions. Sovcit influencers are well within their rights to make up all kinds of gobbly-gookey-garbage pseudo-legal advice.
People who get in trouble are those that follow the gobbly-gookey-garbage pseudo-legal advice.
Chat bots aren’t giving you advice, it’s giving you information.
They aren’t giving you information either. They’re just compiling tokens.
the difference between giving information and giving advice is context. if i know your situation, i am giving advice. if i am just talking about the law in general, i am giving information. the former, i know context. the latter, i don’t.
Let’s swap out a chatbot with a sloptuber on YouTube making up stuff about sovereign citizen nonsense. How about then.
again it’s context. specificity might be a better word? both. are they talking about someone’s specific sitiation or are they talking generalities. does the advice they are giving have context. some rando on youtube, if they’re making up stuff in response to people’s specific questions about their problems and “not” telling them what to do, that can fall afoul of illegal practice of law. if they’re talking about general “well you need gold fringe on your conveyor’s license because admiral keystone q transyldracula said…” in the same way some law youtubers talk about “well here’s how due process works”, it sucks but they have free speech. people are free to mislead each other, unfortunately, just when or if you are relying on those misrepresentations for any transactions it becomes fraud (which is where misleading people becomes a crime). just some examples of the limits on free speech. again, not a lawyer, just have been too embroiled in the legal field all my life.
You aren’t going to get to have it both ways. I promise you, what you are advocating for is such a profound disaster and this whole thing is being astroturfed by tech companies to goad you into limiting your own speech.
what you are advocating for
you are profoundly misunderstanding me
I could see the argument for things that aren’t particularly important, but to continue with the legal example, it seems akin to asking a practicing lawyer a question and asking someone that watched Boston Legal when it aired and can quote James Spader.
Unfortunately, with the potential for a hallucinatory response, anything beyond quite simplistic queries shouldn’t be relied on with more weight than a crutch of toothpicks.
I don’t think you are wrong, but again, thats not the case.
You’re making an argument about speech here.
Lets say you make a fan website based entirely on fine tuned LLM which acts and responds as James Spader from Boston legal. Are you liable if a user of that website construes that speech as legal advice?
If you are willing to give up access to speech so easily, I have almost no hope for Americans in the near future.
What laws like this do is create an incredibly high pass filter to in positions of established power. Its literally suicidal in regards to freedom of speech on the internet.
The right answer is that if you are dumb enough to have gotten your legal advice from an AI hallucination of James Spader, you get to absorb those consequences. The wrong answer is to tell people they aren’t allowed to build fan websites of James Spader giving questionable legal advice.
Presumably such a site would be visually obvious as parody. Having it give jokey answers in as a caricature would be one thing. If you dressed it up as a professional legal advice service for opinions on criminal law from Alan Shore, that could be problematic.
At a certain point of information sharing, we should want a high bar for the ones providing the answers. When asking nuanced questions, we should want for the answer to come from knowledge, not memory. I made an example in this other comment.
I’m not sure I agree with your ‘right answer’ bit. Personally, I’d prefer dumb people to be protected in a similar way that I want the elderly protected from losing their savings from an email scam.
I promise you, the result of this will be unlimited free speech for corporations and their LLMs, with limited and regulated free speech for you. Save or favorite the comment.
It’s the same “protect the children” anti free speech advocacy in a different wrapper, but more appealing to this audience because “llm bad”.
They’re using your emotional response to not liking LLMs as a tool to trick you into giving away your rights.
In your example, say you go to a lawyer and ask legal questions. If the lawyer is not providing legal advise (I. e. taking on the role of being your lawyer and representing you in that matter), they are required by law to express that at the begining so that they will not be held liable because they are a legal professional.
Wikipedia, Google, chatgpt etc are not legal authorities or legal professionals.
There is also no human entity to hold legally responsible if the LLM hallucinates or sites a source that is not factual (satire for instance).
We also know that the vast majority of people who use chatbots do not get the sources they come from.
So. When Wikipedia presents information it is not giving legal advice. That is born out in case law.
The reason it’s dangerous to get legal or health information from a chatbot is the same reason you wouldn’t want to randomly trust reddit.
No lawyers are going to reddit to get help writing legal briefs. We have seen lawyers using LLM’S for that though.
Wikipedia, Google, chatgpt etc are not legal authorities or legal professionals.
Yes. And neither are LLMs or their derivatives.
The reason it’s dangerous to get legal or health information from a chatbot is the same reason you wouldn’t want to randomly trust reddit.
And yet people do, and we accept that as a necessary consequence of maintaining free speech as a principal.
The exact arguments being accepted in this thread are the same which led directly to crackdowns in Hungary, China, and Russia.
If you are okay with limiting and regulating LLMs as a form of speech, I promise it’s your speech which will end up limited, and a very small number of companies will control all speech on the internet. You should stop.
Who’s speach is being limited by limiting LLM’S? Because as a legal entity their speech cannot be infringed because the LLM doesn’t have basic rights in the way that a human does.
So what you’re saying is that you don’t want these companies to be held to any legal standard for the information they output (which is different from reddit because the companies can’t be held responsible in the US under section 230 for what their users write).
The chatbot is the output of the company’s data set and somehow you’re saying the company can’t be held responsible for what that output is and if it’s dangerous because it’s curtailing free speech?
That’s such an interesting take.
I’m gaming out the realistic consequences of what a law will mean. It has nothing to do whatsoever if you approve if these companies or not to try and understand the consequences of what will happen if a law like this passes. You don’t get to pick or choose if the speech is from an LLM or a company that gets limited or from an individual. There is no difference from a legal perspective.
And this law and approach to limiting speech to “protect people” from the stupid consequences of their own action, they aren’t new. And we already know the consequences. Large corporate entities will just get around them or pay an inconsequential fine, and individuals will have their rights curtailed as a result
The entire thread here is falling for an incredibly obvious astroturfing campaign because they associate LLMs with big bad corporations and the real consequences these bad companies have wreaked. But limiting free speech on the internet won’t stop them, what it will stop is our ability to communicate and resist them.
You appear to have gone completely around the twist.
You haven’t shown a logical progression of anything you claim. You don’t point to any current legal precedent, clearly aren’t paying attention to the actual wording being used to draft this bill/law proposal, and are spreading what amounts to FUD.
About the only truthful logical statement you’ve made is that it’s not about whether you like or dislike these companies.
Companies are considered a lawful entity with rights. The supreme Court literally just ruled that LLM’s do not count as the same kind of legal entity because if they did they’d be able to copyright their “work”. So I really do question how you think we go from that to “nobody has free speech because the LLM can’t give legal advice”.
Speech that causes harm has pretty much never been a protected form of speech in the US, even if I were to humor you and assume that an LLM could have the rights to it.
And you mean the “bad these companies have wrought”.
Is the wikipedia responsible for you reading an article about a law and then taking that as legal advice?
Is the U.S. House of Representatives [or any equivalent publisher of the law] responsible for you reading the text of a law itself and then taking that as legal advice?
That’s a totally irrelevant comparison. There is no equivalent publisher of the law to the US House of reps. Nothing the Wikipedia publishes has legal bearing; Everything the house of Reps publish does have legal bearing.
Your objection does nothing to address the issue you raised. Where is the line drawn between “information” and “legal advice?”
Wikipedia and the lawmakers themselves present us with static information that is not specific to us personally or to any particular situation we may find ourselves in, and which generally does not include specific recommendations. I think most people would agree that’s just information, not advice.
If an LLM can be coaxed into saying things like “you should,” advocating specific courses of action for your circumstances, is that legal advice? I think many of us would agree that would be unlicesenced legal advice.
This bill gave us the “best” interaction:
https://bsky.app/profile/badmedicaltakes.bsky.social/post/3mghyg5eufk2m
A Bluesky skeet from @badmedicaltakes.bsky.social:
"Twitter user eoghan:
How dare poor people get free medical advice
<quote tweet from Twitter user Polymarket: BREAKING: New York bill would ban AI from answering questions related to medicine, law, dentistry, nursing, psychology, social work, engineering, & more.>
Twitter user YBrogard79094:
JUST MAKE HEALTHCARE ACCESSIBLETwitter user eoghan:
AI is literally free healthcare. Being a communist must be exhausting"
You can google your simptoms and there probably are some reliable sites but a hallucinating chatbot is a bad idea. Not to mention some people suggested treating covid with chlorine, vinegar etc…
Some horses you can’t even lead to water. Let alone make them drink.
Mixed feelings about this. Let me play devils advocate and say that many Americans don’t have access to these resources at all. Having potentially inaccurate resources might be better than nothing, or is that worse?
it’s worse. In 4D it’s even worser
There are billions being sunk into AI. How much health care could that buy? Your logic only makes sense if AI is free. It’s not.
‘Should I use one teaspoon of salt in this recipe, or two?’
Two is ideal.‘Do dogs like chicken wings?’
Wild dogs regularly hunt small animals like hare or chicken for food.One of these answers results in a bad cake, the other results in a hurt dog. Potentially inaccurate answers aren’t much of a problem when the stakes are low, but even a simple question about what to feed a pet could end with a negative outcome.
Hm, good point. Perhaps the overconfidence AI might provide is even worse than knowing you don’t know.
Having potentially inaccurate resources might be better than nothing, or is that worse?
You pick up a mushroom in the forest and take it home. If you have no information, do you eat it? If something tells you it’s safe do you eat it?
No, misinformation is worse.
the AI devices will just have preambles and disclaimers and word things in ways to refer the user to human resources
If you’re going to be your own lawyer or perform a bit of self surgery, there is no way the AI is helping that situation. Especially if the inherent nature of AI is to validate everything you say.
especially if it’s wrong 20-35% of the time
If you don’t want legal or medical advice from an AI, you can already simply not ask the AI for legal or medical advice. But I don’t want your paternalistic restrictions on what I may ask.
