Hacker News new | past | comments | ask | show | jobs | submit login
Microsoft bans U.S. police from using enterprise AI tool for facial recognition (techcrunch.com)
254 points by coloneltcb 16 days ago | hide | past | favorite | 149 comments



Of course they did. They have to pay separate licensing for the use of DAS – which is developed by M$ and resold from NYPD.

https://en.wikipedia.org/wiki/Domain_Awareness_System


Wow, I had no idea it was this bad. I am not really suprised the lengths American spying goes though. Glad not to be living in New York.

>The Domain Awareness System is the largest digital surveillance system in the world

I wonder how it compares to China and if facial recognition tech is as pervasive in America as it is in China.


There’s London too, totally dystopian.


Most cameras in the UK are private, rarely working well, and usually not networked or easily accessible to the police. It could be better, but the idea of the UK as a surveillance state is seriously overblown outside of maybe a handful of streets.


Don’t forget ANPRs that can track your vehicle all around the country. Doesn’t even have to cross paths with a police car - they’re at most major junctions and all over the motorway


Good. But still doesn't stop the absolute fkwits doing 110mph around the m25 every day; how they get away with it I have no idea.


Because you can't boil a frog by starting at 100°C.

You introduce the surveillance as a way of mitigating extreme, rare occurrences, like terrorism.

You gradually make it common to use when solving major crimes, like murders.

Only then do you start tracking down traffic infractions.

Doctorow calls this the 'Shitty Technology Adoption Curve' https://pluralistic.net/2022/08/21/great-taylors-ghost/


Not really. I got mugged quite close to a tube station in relatively central London and no CCTV footage, so all the cameras are useless.

I was on a jury for a court case where: the incident was caught by a bus camera, but the police waited 6 months to ask for footage, footage is deleted after 1 month and they know this. The shop across the road captured the incident on CCTV but the cop had "tech problems" trying to transfer the video file to his laptop - even then they didn't even bother to sign a document stating what they saw in the video.

CCTV and cops (at least in and around London) are just totally useless. And the gov is useless too, "fighting" crime rather than addressing what causes crimes.

Otherwise, it's not too bad a place to live for a travelling Kiwi, great launchpad into Europe, oh wait they fucked that up with Brexit, too.


London police are pretty much pro dysfunction at this point.


>..., great launchpad into Rwanda. Corrected for you.


> great launchpad into Europe

So pick another country or just fill-in the required paperwork. Wetwipe.


Commonwealth countries in general (I'm in Canada) had a convenient pathway into Europe by way of EU-era UK.

I've lived in several countries at this point, and no, it's not just an administrative task of completing paperwork. Immigration is hard. Do it wrong, and you might find yourself in Rwanda, for example.

And maybe don't call people 'wetwipes' on HN? If you want to hurl insults, well: All the web is yours, except HN. Attempt no landing there.


Partly just a product of its street layout. They need a lot more cameras to get the same amount of coverage that a grid-based city would need, since vantage is so poor.


It's frustrating how Wikipedia phrases unvalidated claims as facts instead of claims. NYC government claiming something doesn't make it true.


In America we get the worst of both worlds: police won’t admit to domestic spying so they can’t use to solve day to day crimes. But they still spy on us, Constitution be damned.

As a result the US has a higher crime rate than many other countries including China. If you don’t trust China’s numbers look at Singapore, which has a population density similar to NYC with an order of magnitude less crime. Singapore is safer at night than NYC is during the day. Why? Cameras. If you commit an offense, you will be caught, without question.


Singapore isn’t safe “because cameras” it’s safe because it has high social mobility, social housing, high levels of education, high levels of income equality and is generally wealthy and much wealthier than its neighbours.

And also it’s a police state where you get sentenced to death for drug smuggling and can be punished for doing drugs in another country while on holiday if you are a citizen, for example.

To the naive, Singapore is a paradise, but once you visit for long enough you realise it’s just a super nice prison and it’s not very fun being with the other lags


William Gibson got a lot of hate for his "Disneyland With the Death Penalty" essay (in Wired), about 30 years ago: https://en.wikipedia.org/wiki/Disneyland_with_the_Death_Pena...


The United States has the death penalty and it’s certainly not Disneyland.

In the other hand calling it “Disneyland” probably oversells how fun Singapore is. It is a fairly boring place in my experience.


"Disneyland" in this context is more about it being a highly managed place with no sense of local history.


USA's local history is genocide and slavery.


This is just straight up Whataboutism now.

Two countries can be bad. It’s not a competition.


> To the naive, Singapore is a paradise, but once you visit for long enough you realise it’s just a super nice prison and it’s not very fun being with the other lags

It's not an American style of life, but I'm sure it's attractive to a lot of people who simply aren't into drugs, partying, or living the high life.


Hilarious, because just like Dubai, nothing applies to the rich. When I was working there we were at Marina Bay Sands.

One night went up to the bar on the roof (god damn the drinks were pricey) clearly ultra wealthy peeps up there, group of very wealthy looking Africans with a magnum of champagne openly smoking weed.


Yes, it's attractive to rich Chinese taking a break from Xi's regime. Others pick Canada or NZ instead.


People complain that you can get fined (the caning is a myth in this case) for spitting your gum on the sidewalk or not flushing a public toilet.

But being a selfish prick is a privilege, not a right. Why do you feel entitled to do shitty things that make your city a worse place for everyone?


Many Americans prefer a non junkie style of life.


This. I'm sure there are lots of things I don't like about Singapore, but the fact that doing and especially dealing drugs is kinda hard and may have death penalty level consequences is a non-issue for me. Cultures that promote drug usage are disgusting and once the drugs comes into your neighborhood there is no way to just ignore it if you care about your safety. I wouldn't care too much if it was just some guys using drugs for fun and that was the end of it, but drugs destroy whole communities and I hate it.


I think that's a USA centric view.

Drugs aren't linked to violence unless you make laws to link them.


Sounds like you haven't met those nice druggies just yet. I don't live in the USA and the law here is extremely lenient for druggies. Yet, pretty much every newspaper I read has some drug related crime in it. Robberies, mindless violence, extortion, murders, damage to properties, lovely stuff. I hate them with passion, and applaud Singaporeans for dealing with the scum appropriately.

Like I said, I would not care if druggies were decent people who just wanted to get high sometimes. However, unfortunately that is not how it works in the reality. They are the worst kind of people and I have zero sympathy for them after all the shit I've seen them do.


You talk about meeting and then about newspapers? Have you met them yourself?

> people who simply aren't into drugs

...are we still having to go over this?

There are no such people. Everybody does drugs, unless you adopt a definition of "drug" which is designed specifically to accommodate this assertion, rather than apply in some useful way to reality. It's difficult to even create a definition of 'drug' which convincingly excludes survival necessities like water and oxygen, but remains otherwise consistent.

There are some drugs which are, for reasons varying from racism to capriciousness to an intemperant desire for greater cartel income, prohibited to varying degrees in various jurisdictions.

You can go to Singapore and consume sugar, coffee, nicotine, alcohol, and many other drugs while enjoying the tacit endorsement of your behavior by the local state apparatus. You can also consume cocaine, opioids, and plenty of other drugs so long as you pay black market rates and do so shielded by privilege so as not to run afoul of an investigation. You can even consume cannabis, though of course it is very difficult to smuggle, so you pay a higher premium.


You know what we mean by drugs. Not coffee, not alcohol.


No, I really don't know. Do you just mean drugs which the state prohibits?

If so, isn't that begging the question? If a person wants to go to a place where drugs are prohibited, and "drug" is defined as a substance which is prohibited, then doesn't every jurisdiction qualify?

If a person "simply isn't into drugs", with "drugs" defined by the local state, isn't that person just in favor of the state-prescribed diet? And it really has nothing to do with drugs?

If such a person goes from a place where drug X is prohibited to one where it is not, does their preference suddenly change?

One better: if a person goes from a place where drug X is prohibited to one where it is compulsory, do they suddenly fall in love with the drug?

Of course not. Because nobody's preferences work that way. Ergo, there is no actual real human who meets the critera, "simply isn't into drugs" upon close examination.

Sure, there may be people who are pro-prohibition, but I imagine nearly all of them are in a position to financially gain from it. In terms of outcomes, it's horrible policy, especially if a target is reduction of death and disease from drug use.


Why do you consider weed to be a drug and not alcohol? Alcohol can be far more damaging both to the person and those around them. Hell, caffeine probably fucks up your system more than weed too.

So why are they not considered drugs? Is it because of literally the comment you responded to?


I went to Singapore for a week of work a few years ago. >high levels of income equality

Oh yes, I'm sure all of the Indians crammed into the back of a caged ute on their way to building sites have income equality. Saw loads of stuff like that there.

Or the fact that being gay is still illegal, didn't stop me "breaking the law" the second day I was there.

I put Singapore into the same category as Dubai; it's all a facade and all can be conveniently ignored with a veil of money.


When the choice is a civilized prison or a jungle, the prison isn't so bad.

How many children were murdered (by guns or drugs) in school in Singapore last year?


> To the naive, Singapore is a paradise, but once you visit for long enough you realise it’s just a super nice prison and it’s not very fun being with the other lags

Oh come on, your yardstick is completely ill-calibrated. Among polities that

- didn't exist as independent polities until post-WWII

- were not modernized/westernized/democratized until post-WWII (thereby excluding NZ)

you'd be hard-pressed to name one that you'd be rather born as a random citizen in. Perhaps another of the Asian tigers, like TW. Presumably you have in mind, as Gibson probably did, a country that has had a longer history of independent or democratic rule and economic development.


The claims made in your first paragraph are undermined by your third paragraph.


the first paragraph did not claim it was not a prison, it just claimed it was safe not because it was a prison but for other reasons.

The third paragraph claimed it was a prison but it did not say the safety came from it being a prison.

the third paragraph and first paragraph are not in conflict.


The fact that it is a prison with much surveillance has little to do with its low crime? This is not a credible belief.


I wasn't saying it was wrong or not - I just said the two paragraphs were not inherently contradictory.


That’s why I said “undermined” and not “contradicted”.


I know it’s not true in American prisons but generally not much crime occurs in prison. If Singapore is prison like then it seems that this would at least be a contributing factor to its lack of crime.


Not that US institutions don't lie, but our bureaucracy and government oversight systems come with the benefit that many of the things we track and document can be verified. Other countries simply lie. The numbers you get are not a rough approximation of reality; they're simply part of whatever story that country wants to tell.

Crime rate is a statistic for which a majority of countries provide numbers that are completely dissociated from reality.

You can, of course, take the numbers seriously, as if the statistics are being published in good faith. Unless you have some sort of independent oversight, however, that isn't beholden to or biased by the country being assessed, then taking those numbers seriously is probably a silly thing to do.

The US gets lots of independent verification and validation of crime statistics. They're frequently analyzed at local, state, and federal level by journalists, students, activists, authors, and government officials. At every level an official number is published, it gets challenged, so there are incentives keeping the politicians and bureaucrats honest. They get slammed when they get caught lying, and they get caught lying because the public and the media keep track of things and demand accountability.

Some stuff, like total officer involved shootings, dog shootings by officials, abuses of power, and things of that nature, don't get publicly disclosed much of the time, so there are gaps in what we know and what officials are required to disclose.

The US isn't perfect, but you can get pretty good numbers that actually correlate with reality. Even other western countries don't always have trustworthy reporting and accounting for government actions. The best you'll ever get is a glowing narrative.


You're not wrong, however when it comes to NYC statistically it is the safest in the US. However this is mostly looking at murder/capita.

It does not account for the 1000s of illegal small assaults that happen daily. Anyone spending time in NYC will eventually run into this.

But then you get the one person who says "I've never been mugged" or "I've never been punched" or a journalist vists for a day and says it's all fine.

Meanwhile there is a lot of stockholm syndrome of what QoL people put up with on a day to day basis.


Singapore has very low levels of corruption. Even lower than in the United States. You can choose to dismiss them, but I think their numbers are reliable.


This post should be stickied at the top of this topic.


Such a long text for "their numbers are better than ours, they are clearly liars"


Same with national ID and our fear of “papers, please”. We have all the downsides—constantly having to provide ID to everyone, government can trivially access all kinds of tracking data tied to that—but none of the benefits of an actual national ID because we have to pretend we don’t have one (we do, it’s just 80% privatized and a massive liability and inconvenience for citizens in ways that it doesn’t need to be)


> we do, it’s just 80% privatized and a massive liability and inconvenience for citizens in ways that it doesn’t need to be

I am guessing to you are referring to how multiple private organizations can just poll for my social security number and I cant do anything about it? Yeah...


Yep. It’s a pain in the ass to deal with from our end, to put it mildly, but easy to piece it all together on the other, so it’s barely an impediment to bad actors. Since there are functionally no restrictions on government using parts of that system to grab all kinds of data from private sources, it’s a national ID connected directly to a crazy-powerful dragnet spying system, too. But inconvenient, insecure, and very hard to gain oversight of.

All the bad shit, none of the good. We may as well just have a national ID, it’d be less-bad than what we have now and might provide a jumping off point for making it a lot less-bad.


The irony of individuals freaking out over the notion of what the wildly underfunded and largely indifferent federal government might do to collect data when private industry is surveilling basically every aspect of their existence for most of every day...


Private industry can't directly send someone to prison.


Soon: "our AI predicts that individual X has a 90% chance of committing a violent crime within the next semester"


Private industry sells your data to the government to do that.


"wildly underfunded"? Do you really believe what you just wrote here?

What private industry surveillance is mandatory? FFS.


User "forgetfreeman" has not actually looked up the budget of the US government.

> constantly having to provide ID to everyone

I live in WA and for the past 5+ years I've only ever presented my driver's license to TSA agents at the airport.

...unless getting carded at the store for buying beer counts?


Doctor, dentist, employers, educational institutions. [edit] all financial institutions, lenders, et c

Unless you’re buying everything with cash, your purchasing history will be connected to it, too. Where you drive (license plate scanning and data-sharing is widespread). Facial recognition in stores means paying cash might not even help (seen the ones that highlight your face on a monitor when you walk in the store?)

But we have to pretend we don’t have a national ID system any time it might be convenient to a person to have that.


Right. I thought the parent post was talking about being challenged by government officials/police ("papers, please") on a regular basis - and not about private-sector use. A misunderstanding on my part.


I literally had to present ID this morning to buy cold medicine. Had to present ID lastweek to enter my kid's school then had to present ID to their pediatrician.


It's sad how in the state of Washington I have become accustomed to drawing my ID out on every transaction that involves confirming that I'm over 21 whereas cross the border into Idaho there's not a single time I have ever been asked to provide my ID in over 10 years. Clearly in Idaho they can take one look at me and go like well yeah clearly but such visual social technology is beyond the pale in the great creepy state of Washington. I'm a bit sad to think of what the children of today's children will be like when I am very old and they are very young. Maybe one day nobody will know what it was like to live in a society that was able to just tell things by looking at it.


At least two of those instances make perfect sense. Meth's a thing and we don't like unidentified individuals fucking around on school grounds these days.


I think Singapore whipping the shit out of criminals probably has a lot to do with it. Not just Singapore, but most of the rest of Asian countries as well. South Korea treats their prisoners so harshly that the US military has to have a special agreement in place to ensure that US servicemen imprisoned for crimes by South Korea actually get fed more than a starvation diet and don't have to do hard labor: https://www.stripes.com/migration/for-u-s-inmates-in-s-korea...

Of course South Koreans know better than American soldiers not to commit crimes in South Korea, because prison there is so awful. Unsurprisingly they have a much lower crime rate than America.


The big downsides with horrible prison are that criminals will have a huge incentive to do anything to not get caught and convicted, including violently resisting arrest, shootouts, targeting witnesses and so on. It is mostly effective as a deterrent against financial/winning-relates crimes, but mostly ineffective against most other types of crimes. Plus, horrible prison is much less likely to actually reform the person in question.

Statistically, the Norwegian system with "holiday prisons" works well, with very low recidivism rates for most types of crime.


Norway's intention homicide rate is six times greater than Singapore's.


That’s why execution is the ideal solution, not prisons. During a brief transition period you may have to deal with violently resisting arrest and the like, but eventually you get to the point where all the criminals are dead.


本当の殺人鬼だな、キラは.


Hah! It had some flaws, but I did enjoy that manga.


Studies show that the severity of the punishment has little impact crime rates -- the odds of getting caught & punished are much more significant.


> Singapore is safer at night than NYC is during the day. Why? Cameras. If you commit an offense, you will be caught, without question.

There's more to it than that. A sheisty will defeat a camera.


Not when you have a lot of cameras, right? You still have to put it on somewhere without being seen, and you would have to swap all clothes so you can’t be tracked by checking tapes of other locations for your clothes before you put it on.


Is it the catching or the imprisoning NYC and many other cities are struggling with in current year?


American criminal subcultures treat going to prison as almost as a rite of passage, rather than something to be avoided at all costs. Sentences are too short and too easy.


Yeah I would want to better understand how they arrived at that statement.

When I was in Xinjiang there were like 5-10 cameras at every intersection. Surely NYC isn't at that level?


I don't know about NYC but it's common to see at least 4 cameras per lighted intersection where I am (small city in the PNW). They're replacing the loops used to sense vehicles, but I'm pretty sure they're also remotely accessible.


Not only are they remotely accessible. There publicly accessible (not all of them) https://kingcounty.gov/en/dept/local-services/transit-transp...


I don’t think those are necessarily recording cameras. They could just be radar or video vehicle detectors to let the light know if a car is waiting.


You had no idea it was so bad? You do know how mobile phones work and what is tracked don't you?


> 9,000 CCTV cameras, owned either by the NYPD or private actors

I wonder what kind of person volunteers their camera for the surveillance apparatus.


The business owner who might have had their fill of break-ins, people pooping in front of their store, et al., for one. I'm sure "private actors" is not all just Ring doorbells.


Businesses point cameras at the public for the sole purpose of surveillance. I'm not talking other businesses, I'm talking business built to sell and launder surveillance footage.


Yay, even licensed to the Brazilian National Police (that doesn't exist). Did they mean the Federal Police?


It seems to be a mistake. From the original article which Wikipedia is citing:

> "[...]Others, such as the Washington D.C. Metro Police, the Singapore Police Force, and the Brazilian National Police have purchased the DAS software from Microsoft, our software developer, and have used it to secure high-profile governmental and cultural sites, the 2014 World Cup, and the 2016 Summer Olympics. Microsoft has agreed to give New York City 30 percent of the revenue it derives from selling the software to other jurisdictions (Parascandola and Moore 2012);"

But "Parascandola and Moore 2012" refers to another article that doesn't say anything about existing uses outside of NYC. (Using Archive because the site has a Geoblock).

https://web.archive.org/web/20210812131624/https://www.nydai...

EDIT: Just removed the paragraph from Wikipedia.


It reads like literal "pre-crime."


A machine learning algorithm known as Patternizr is included in the DAS, which connects potential criminal suspects to other unsolved crimes in order to speed arrests and close old cases.[20][21] The algorithm is trained on a decade of historic police data of manually identified crime patterns.

Potential criminal suspects. Who needs civil liberties, anyway?


You are criticizing a unsourced claim on Wikipedia, not validated information about Patternizer.

Regardless, how do you think crimes are investigated? Is it your theory that suspects must volunteer themselves before being investigated?


So no more "Copilot can make mistakes" kind of disclaimers? Patternizr nevr maks mstakes?


They are not banning them from using their AI tool. They're banning them from the currently-offered licensing structure. This is a profit-grab, not an ethical restriction. It is of the same vein as Sam Altman going to congress and spooking them in hopes of regulating AI so that they are the only ones allowed to build it.

I wouldn't be shocked if we also saw a similar move to ban hospitals, schools, etc. from using the standard enterprise license under the guise of "we need to provide you with oversight for your vulnerable populations", and then charging them a hefty surcharge on top of the standard enterprise rate.

"Responsible" AI is more profitable than standard AI - regardless of the nebulous nature of what "responsible" means.


Spot on. I'll explore whether this nit is worth picking:

> "Responsible" AI is more profitable than standard AI - regardless of the nebulous nature of what "responsible" means.

/s/regardless of/in part owing to


American law enforcement openly disregard the foundations from which the country was built upon yet pushes other countries to adopt ideas which it abandoned but won't admit it out of humiliation--that which we call "democracy" comes with the very hooks and strings it tells other countries to abandon.

Just take a look at the countries that credulously adopted "democracy". They are not in a good shape and will probably never find itself out of the moving goal post that Western countries like to place on "lesser" countries.

America is addicted to these 3 things: Oil, authoritarian labour, drugs. It's law enforcement is not meant to police those addictions at all but rather to ensure its continued uninterrupted supply even if it means violating its own principles that made the country great (about 40+ years ago).

It's no wonder many Americans are yearning for past glory.


Yes, the past glory of Legally mandated racial discrimination, and a war on Black people and Hispanics on a pretext of a War on Drugs.


This is excellent. There will be blood on our hands if we allow our software to be used to more closely surveil and police people. Not only because the software as it currently exists is flaky, but because of how it will enviably be used. Certain communities will have it deployed against them more, parallel construction will hide when technology is used in violation of rights, police and juries will believe when technology has a "hit" that it proves guilt and not understand the caveats, etc etc.

Some Microsoft employees have been calling for such limits for years [1, 2]

[1] https://www.geekwire.com/2019/microsoft-github-workers-prote... [2] https://www.cbsnews.com/news/microsoft-workers-asking-drop-p...


The NYPD has had a system built by Microsoft for 15 years:

> Beginning in 2009, the NYPD, in partnership with Microsoft, built a powerful counterterrorism and policing tool called the Domain Awareness System (DAS). The DAS is a central platform used to aggregate data from internal and external closed-circuit television cameras (CCTV), license plate readers (LPRs), and environmental sensors, as well as 911 calls and other NYPD databases. The DAS uses an interactive dashboard interface to display real-time alerts whenever a 911 call is received or a sensor is triggered. The DAS also in- cludes mapping features that make it possible to survey and track targets. The DAS was originally built to support the Lower Manhattan Security Initiative (LMSI) – a public-private partnership – but has since expanded to cover the entire city, giving NYPD personnel direct access to thousands of cameras owned and operated by private orga- nizations. Until the development of the mobility platform and the mobile DAS system described below, the full capabilities of DAS have only been available to the Counterterrorism Bureau and a few other specially trained officers on desktop computers. As the NYPD upgrades its network, access to all DAS capabilities and resources will be expanded to all NYPD’s commands.

https://www.nyc.gov/html/nypd/html/home/POA/pdf/Technology.p...


If it's like what they did with the military, they're just trying to push them into a more expensive licensing scheme.


"In January, reporting by Bloomberg revealed that OpenAI is working with the Pentagon on a number of projects including cybersecurity capabilities — a departure from the startup’s earlier ban on providing its AI to militaries. Elsewhere, Microsoft has pitched using OpenAI’s image generation tool, DALL-E, to help the Department of Defense (DoD) build software to execute military operations, per The Intercept."

Hm... to my eye, this just seems to be a great spin on essentially pushing cops or their vendors to sign (more expensive, I imagine) independent service contracts under the guise of oversight. Their principles seem to be as genuine as they were in their plea to the US government to cede control of LLMs, generally, to the big corporate players "for safety."


OpenAI has blatantly ignored most of its principles (which Elon called out surprisingly well). Insane to me that people ignore that.


Seriously.

> Insane to me that people ignore that.

Yeah for sure, but I don't think this is a super representative crowd, though. People uncritically frolicking through all of these developments with AI-enhanced rose-colored-glasses glued to their faces are over-represented in the SV universe. I know a lot of non-tech-focused folks that are casting a very wary eye towards our impending incredible AI-powered corporate utopia. The only similarly optimistic people I know outside of tech are the devastatingly credulous fellows still trying to figure out how to recoup money on all of the NFTs they bought, and they seem to be much better at generating headlines than societal changes. But maybe I'm the one wearing the rose-colored glasses, here.


Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ?

Feels like they're just protecting their ass while keeping overselling their product.

Not that law enforcement should use AI irresponsibly and without boundaries, but I'm not sure it is the responsibility of a company to make that kind of call.


Presumably if I have a service that I think is inappropriate for a use (for whatever reason, but let's say false positives) I can in fact limit what kind of use cases area allowed.

I could argue that my tool used wrong presents a legal risk to ME because I let law enforcement do some terrible things that I know don't work.

Semi-related, most cloud services do not allow crypto mining, the reason there being it's not cost effective. So whatever mining does happen is stolen credentials, or some inside company job where dude is using company resources for personal gain ... pretty much all illegal activity. So they ban it and I believe actively try to detect / aggressively disable it.


I think what it's about, is they can squint a little bit and see a near future where there is massive class-action suits against Microsoft for all the harmful uses which inevitably flow from a police department uncritically using hallucinating AI for their day to day work.


You generally can't force a party to enter into a contract with another party that they don't want to.


Utilities and carriers can’t just pick and choose customers totally. For example, your electrical utility cannot deny your service based on your political speech or profession or whatever. Even if they are privately owned.

Personally, I think companies above a certain size should not be able to deny customers because they are effectively like public agencies or utility services due to their size. Banks, big tech, large social media platforms should be regulated like utilities in my opinion.


When it comes to wanting big companies to follow rules (a noble pursuit) many wish that common carrier status could be imposed on a business.

But remember why common carrier status exists. Imagine I rob a stagecoach and lie low until the next day, when I catch a train to the next town over.

Is the train engineer my accomplice in the robbery? Or my accessory after the fact? Is the train company?

He and it might be, except that the train company “holds itself out” as a transporter of goods and services open to all. In exchange, the law grants the company that holds itself out as a carrier for all the freedom not to worry about whether carriage is a criminal act for the customer.

But without holding one’s self out as a carrier for all, there is no common carrier status. You cannot hold out a company as sufficiently large; they must hold themselves out as wanting this status.

When I see people clamor for social media platforms, I get doubly confused, because not only have these companies never held themselves out as common carriers, they would take nothing they do not already have (under 230) from doing so.


> Is the train engineer my accomplice in the robbery? Or my accessory after the fact? Is the train company?

No, even without common carrier status. Being an accessory to a crime is much more narrow than "unknowingly conveyed a criminal during routine operation of a transportation company".

That broad of an interpretation makes everything being an accessory to the crime. The federal government created and backs the bills the thief stole, so they helped. The city built the roads they used, so they help. Old Mister McCollem down the lane stopped walking so the thief didn't run them over, which might have helped.

> But without holding one’s self out as a carrier for all, there is no common carrier status. You cannot hold out a company as sufficiently large; they must hold themselves out as wanting this status.

This is not true, you're misconstruing the difference between companies that had mens rea in transporting illegal goods and companies that don't.

Common carrier is an involuntary status. The government decides whether a company is or is not a common carrier and is obligated to provide services to anyone that asks (basically). Common carriers are almost categorically excluded from liability for the goods they transport because they are effectively incapable of mens rea. They did not choose to transport those goods, therefore they could not have had criminal intent in making a choice to provide service.

Companies that are not common carriers _do_ have a choice when deciding whether to provide service, and thus are capable of having criminal intent when making that decision.

Inversely, companies that choose to act like a common carrier _do not_ receive the same protections as an actual common carrier. Their decision to provide service to everyone without asking questions is still a decision, and can meet the mens rea bar.

The most well known cases I can think of here would be The Pirate Bay and Silk Roads. Both behave as if they were common carriers and will sell/convey almost whatever you want without questions. They do not receive immunity for not asking questions, because the intent of that policy is to allow (encourage, really) illegal usage.

On the flip side, the Post Office/UPS have indemnity for conveying drugs bought on Silk Roads because they actually are common carriers. They didn't decide to just not check packages, they are legally prohibited from invasive package checking and denying service to customers they find sketchy (would need a fact check on the prohibition of checking packages; I believe there are limits on what they can check, but I don't know offhand what they are).


If you think common carrier status is involuntary I’m not sure what to tell you except that one of the elements of the common carrier test is that the common carrier “holds themselves out” as a carrier for all. That’s a voluntary act.


Of all the things that could be considered a public utility, Azure OpenAI Service ranks somewhere between Cabbage Patch dolls and Crystal Pepsi. Not only can everyone live without it, almost everyone on the planet does live without it.


But utilities also have those rules because they are granted de-facto monopolies in their geographic areas.

It's not like I can get electricity from another provider if there is literally only one in my area. And what they're providing (water, sewage, electricity) is deemed essential.

Even with banks, if one bank rejects me, I can (hypothetically) go to another one down the street.


Companies can be damaging and anti competitive even if they aren’t total monopolies. Being an oligopoly or having a significant portion of market share is enough. Keep in mind many of these companies have more users than most national governments do - and they have soft power on a similar scale.

Another angle is network effects - social media companies face reduced competition due to this and so they too should be regulated above a certain size since there isn’t enough competition or choice there, because the value of access to those particular platforms is very high due to their size.

> Even with banks, if one bank rejects me, I can (hypothetically) go to another one down the street.

What if they all reject you though? For example what if all banks decide they are better off debanking someone to stay in the good side of politicians who might punish them otherwise (like via regulation)? Some of these services are fundamental and should be forced to provide service no matter what.


microsoft's enterprise AI tool isn't a utility though, and they're not banning police on political or moral grounds.


> microsoft's enterprise AI tool isn't a utility though

It’s part of an infrastructure as a service platform, which is why I view it as utility. But I also am saying we need to broaden what we call a utility for the modern world. Not having access to large platforms (like social media) or centralized services (banking) or large companies (which have lots of market share and are one of a few options) is very damaging. What are your thoughts on that?

> and they're not banning police on political or moral grounds.

How do you know this? This seems like it would almost certainly be the result of pressure from political groups.


> It’s part of an infrastructure as a service platform, which is why I view it as utility.

They haven't securely broadened it to the internet or mobile phone service, and you want them to broaden it to individual privately-produced software packages. So the police can't be refused as a customer.


I think in this case, a bigger factor would be this is a 'we will not attest to accuracy in court'.

As an odd sort of comparison, this is why you can't just take a magic 8 ball into the courtroom and shake it; 'for amusement only'


> Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ?

Companies can choose their clients in a non cyperbunk world too.


Corporations can make any judgement call about who can or should use their product.

Though it does seem rare in the US for companies to prevent others from using their product based on ethical or moral reasons, but more should.


US banks don't do business with the porn industry pretty famously.


That is not due to morality though - that is due to the insane level of chargebacks and fraud revolving around payments within the porn industry.


Except it is. High shrinkage can be baked into the business model and digital media is a high margin business that can tolerate it. Other industries with high fraud rates don't get cut out wholesale.


Exactly. Mastercard, Visa & Discover were perfectly happy to process PornHub payments for years - until enough external political pressure was applied. Religious special interest lobbying groups like like Focus on the Family and NCOSE (formerly known as Morality in Media) were celebrating when they stopped, as were Republican politicians like Josh Hawley (https://twitter.com/HawleyMO/status/1337111830976753672).


> until enough external political pressure was applied

This doesn't pass the sniff test. You have been able to purchase porn for decades in various mediums - why would a website suddenly invoke the ire of the "morality police"? Even more-so when porn is freely available in almost every case when it comes to the internet.

Really though, it has nothing to do with morality - it is straight up the risk profile of those websites. There's even a movie about the start of the online paid-porn industry and all the stuff they had to do to figure out how not to bleed money on chargebacks etc.


> You have been able to purchase porn for decades in various mediums

Have you? Or has it been patchwork and limited and volatile, with people being fined and jailed fairly arbitrarily? There were periods in the 80s and 90s where people were going through pornographic tapes and removing the swearing.

> suddenly invoke

Suddenly, after decades. Is the adjective just to make it sound more unlikely?


What about Visa and Mastercard blocking Wikileaks donations? Should that be allowed just because they are private?


Not the other guy, but I'm pretty much OK with that. Honestly I'm more OK with them blocking payments to quasi-illegal operations than I am with them blocking payments to presumably legal adult content. Ultimately, in both cases, there other ways to make those payments. Hell I'm more annoyed that I had to use giftcards to buy Minecraft for my kids back in the day, before Microsoft became their parent company, because of some weird financial law in my state.


> Hell I'm more annoyed that I had to use giftcards to buy Minecraft for my kids back in the day, before Microsoft became their parent company, because of some weird financial law in my state.

I'm curious to know more about this. I'm not aware of any laws in any state (assuming the US) where it's illegal or forbidden from purchasing video games using a credit card, or even a debit card.


> I'm more OK with them blocking payments to quasi-illegal operations

I know cases have been brought against Wikileaks and Assange, but in my opinion they broke no laws. They acted within the bounds of journalistic freedom. I don’t understand how people are OK with news outlets leaking things like Trump’s tax returns but think that freedom of the press works differently in other circumstances. Why would it be different case by case?


Companies can decide who they provide service to, and can give advice about who "should" use their products. But I don't think Proctor & Gamble has the legal authority to stop you from using Tide laundry detergent that you legally purchased from a grocery store or on eBay.

The current era of tech is all about disguising web services as "products" you can "buy" (most recently exemplified by the Humane AI pin and Rabbit R1), so I can see how there'd be confusion, though.


Forcing companies to sell their surveillance-enabling product to the police seems pretty cyberpunk-dystopian to me.


> Wait, is that a decision a company is allowed to make ? Is it the beginning of cyberpunk ?

Yes. Freedom of association is literally part of the first amendment. The only limits are around protected classes.

How in the world is forcing a company to do business with someone a LESS dystopian solution than allowing them to decide who they do business with?


FYI, Microsoft now says (we contacted before publishing) the policy we cited "contained an error" and was intended to only apply to facial recognition, and has been changed in the official code of conduct. We've updated the headline and body to reflect this change. A mod might want to change the HN hed too.


No one has pointed out that the AI companies are being sued for 'stealing' vast troves of copyrighted data, and the case is still in progress.

If the police departments are complicit, then does copyright stand to lose de facto as AI will only be increasingly implemented while the case slogs through the courts?


One bare minimum way to not get your data "stolen" is to not personally make it freely available on the internet.


How exactly are they going to stop the NYPD from using this service? Just not register it in their subscription as a service? And does it really matter? If I can use Azure's cognitive search and host my own model that doesn't stop me from using Azure for the same purpose.


Using it unofficially through unauthorized workarounds brings doubt into the chain of custody and investigation integrity.


I'm not sure the courts would consider it a workaround, it's a legit way to host your own model.

https://learn.microsoft.com/en-us/azure/machine-learning/how...


Clearview AI created an unregulated facial recognition system and let individual cops use it for free as a "trial" Their superiors claim they had no idea https://www.abc.net.au/news/2020-07-13/afp-use-of-facial-rec...


when cops and DAs use parallel construction to build a case, "integrity" is a mere technicality.

https://en.wikipedia.org/wiki/Parallel_construction


Oh no, they will have to use parallel construction, something they TOTALLY don't have experience doing.


You can't stop someone from misusing your technology by updating your terms of service, just like you can't stop a criminal by making their behavior against the law. But I don't hold Microsoft responsible for making the police behave a certain way, or for preventing violations of the rules from happening. As you say, it's just not possible. I'd be satisfied if Microsoft consistently shut down any attempts to use their technology in a systematic way, once they discovered a violation of their terms.


Is this just PR stuff? Obviously enterprise tech companies (Google/MS) are going to be working with States. These State have to enforce the social contract so you are going to have blood on your hands.

I don't really have skin in the game, I'm at the whim of these parties.


I suspect it's legal. The focus seems to be on face and speech recognition and I am sure MSFT's internal metrics on the AI's ability to recognize non-white faces/accents/etc are very bad.

Even with LLMs, the existence of lawyers unknowingly using ChatGPT confabulations in court suggests that police officers will do the same with investigations or surveillance. A cop who takes Sam Altman's dumb tweets too seriously might think Azure's GPT-4 service can look at an evidence report and apply its AI Magic. Probably not something Microsoft wants to defend in court.


This is a great opportunity for open-source models to get a foothold in this lucrative market.


IMHO, speech-to-text to LLMs is borderline spycraft already.

“Are you wearing a wire?” The answer doesn’t mean merely recording the conversation.

Now it could mean, a personalized “analyst” determining sentiment, correlating events/evidence/facts/(hallucinations)/building a prosecutors case…in real time!


This seems like a good thing. I would not trust US police to hold themselves to literally any standard of ethics as far as automation of this sort is concerned. I can easily see departments ignoring the outputs that dont fit into their agenda while running running with the outputs they do like.

I would like to see more companies restricting what contracts theyll take based on ethics.


Unfortunately that’s not what’s happening here. The police still have access to this information and tooling. They just have a different license agreement and access point that allows Microsoft to extract more tax dollars.


There is probably an actual reason for it other than corporate greed. I'm guessing they have to silo the data separately on their cloud and that costs more. It's the same reason healthcare applications often cost more to host, they can't intermingle data in cloud instances and have to insure that it's only hosted in the US and what not. I imagine having to deal with legal procedures ensures a lot of additional time/money compared to other customers.


Oh I would certainly bet there's a lot of reasons it costs more money too. I'm just being reductivist and pessimist about it!


AI is a giant liability shield. You can blatantly steal shit and profit of it. Of course enforcement agencies are going to have a field day with it. Announcements like this are just a pretense that they aren’t already counting the money.


It is dangerous to allow groups of any sort to be excluded from licensing. This sort of behavior is far too easily weaponized against minority political, opinion, & affinity groups. While I'm not keen on the police having these tools in their hands, who else will be next? Who will be squeezed out of the economy because everyone who provides a particular sort of software or service decides to enact the same policies? This is why software should be sold, not licensed — a word processor should be treated no differently from a loaf of bread.


"First they came for the Police state and I said nothing..." lmao




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: