On April 1, the Italian government banned ChatGPT, an artificial intelligence system that generates human text and computer code. ChatGPT is trained on vast amounts of data scraped from the Internet, and Italian authorities were concerned that this constituted a breach of privacy for those who owned the data. The ban remains in place while it investigates further.
But this ban has unintended consequences. One of the big questions about ChatGPT and other so-called big language models is how people start using them to improve their productivity (or replace other people entirely). The potential seems clear, but no one knows how ChatGPT already filters in the workplace.
Enter David Kreitmeir and Paul Rashki of Monash University in Australia, who realized that Italy’s ban inadvertently created the perfect conditions to find out by looking at patterns of work before and after. It turns out, they say, that ChatGPT is not just widely used in at least one profession, but that productivity in that industry has come to depend on it.
A matter of coding
The field of work in question is computer coding. There is plenty of anecdotal evidence that developers make extensive use of ChatGPT to develop, write, and debug code, but evidence beyond that is hard to come by.
But Kreitmeir and Raschky realized that if programmers in Italy had come to rely on ChatGPT, the signs should quickly become apparent when access was shut down. It turns out that developers’ work patterns are recorded in various ways by GitHub, a widely used online software repository that also manages version control.
So they pulled data from GitHub showing the hourly coding of 8,000 programmers in Italy and two other European countries before and after the ban.
The results are something of a surprise. “We find that the ChatGPT ban reduced the production of Italian GitHub users by about 50% in the first two business days after the ban began,” say Kreitmeir and Raschky. “Baseline levels returned to normal levels thereafter.”
Researchers say the return to normal is likely the result of coders finding ways around the ban, which is easy to circumvent by using a virtual private network or Tor Relay to access ChatGPT in other countries.
Decline and fall
Indeed, the researchers noted an immediate increase in Google searches for “virtual private networks” the day after the ban.
“Our findings support the idea that the significant but short-lived drop in production after the ban was the result of Italian consumers seeking and successfully finding ways around blocked access,” they say.
This is interesting work that suggests that ChatGPT is being used much more widely than expected. “We interpret these findings as the first evidence that ChatGPT is already being actively used for highly skilled tasks in the economy,” Kreitmeir and Raschky conclude.
Given that ChatGPT was only launched last November, this may just be the first trickle in a tsunami of evidence.
Reference: The Unintended Consequences of Digital Censorship – Evidence from Italy’s ChatGPT Ban: arxiv.org/abs/2304.09339