O brave new weirdness

Imagine being hired by a city to serve as a business liaison, someone charged with providing information to business owners about regulations, laws, programs and services available to them, city codes and the like. Important stuff.

“Thanks for the job,” you say, “but I want to make one thing clear: My answers to the public’s questions may be incorrect, harmful or biased, and I have limited knowledge of the world beyond local business, which may impinge on my judgment.”

The city says, “No problem!” and keeps you on the job even after you give employers false information about tips and wages, worker protections or consumer and tenant rights — things that can injure people and lead to lawsuits or prosecution.

If you still have your job after all that, you are probably an AI chatbot.

New York City launched its “MyCity chatbot” last fall, powered by Microsoft. Five months later, The Markup reported the software providing bad advice across the board: Sure, bosses can dip into the tip jar; you can fire workers if they complain about sexual harassment; you can discriminate against Section 8 tenants; your store can stop accepting cash payment. It even told an Associated Press reporter that restaurants could serve cheese with rat bites.

The MyCity chatbot is an outright hooligan, encouraging employers to violate numerous local and federal laws. On the other hand, it’s not being paid salary and benefits.

And the city is not taking it down. The site continues to dispense bad advice and even posts a warning about “incorrect, harmful or biased” answers, yet Mayor Eric Adams said it will stay up so it can “iron out the kinks.”

We have been trained to call this technology “artificial intelligence,” yet we have not built intelligence. We have built a parrot with access to massive data sets. It can be a useful tool, but we should not be regarding it as intelligent or relying on it as a reasoning entity. We must think clearly and carefully about this, especially when it comes to displacing human advice.