[ad_1]
Hey guys, welcome to TechCrunch’s regular AI newsletter. If you want this in your inbox every Wednesday, sign up here.
It was very easy to create a compelling soundbite for Kamala Harris on Election Day. It cost me $5 and took less than two minutes, illustrating how cheap and ubiquitous generative AI has opened the floodgates to misinformation.
Creating a deepfake of Harris was not my original goal. I was playing with Cartesia Voice changera model that transforms your voice into a different sound while maintaining the original tone. This second audio can be a “clone” of someone else – Cartesia will create a digital audio double from any 10-second recording.
So, I wondered, would Voice Changer turn my voice into Harris’ voice? I paid $5 to unlock Cartesia’s voice clone feature, created a clone of Harris’ voice using recent campaign speeches, and selected that clone as an output in Voice Changer.
It worked like a charm:
I’m sure Cartesia didn’t intend to use her tools this way. To enable audio cloning, Cartesia requires that you check a box indicating that you will not create anything harmful or illegal and that you consent to recordings of your speech being reproduced.
But this is just an honor system. In the absence of any real safeguards, there’s nothing stopping anyone from creating as many “malicious or illegal” deepfakes as they wish.
This is a problem, it goes without saying. So what is the solution? Is there one? Cartesia can implement voice verification, as some do last I did the platforms. But by the time that happens, a new unrestricted audio reproduction tool will likely emerge.
I spoke about this very issue with experts at TC’s Disrupt conference last week. Some were supportive of the idea of invisible watermarks so that it would be easier to tell if content was generated by artificial intelligence. Others pointed to content moderation laws such as the UK’s Online Safety Act, which they said could help stem the tide of misinformation.
Call me a pessimist, but I think those ships have sailed. We’re looking at it as, as the CEO of the Center for Countering Digital Hate, Imran Ahmed, put it, “a perpetual bull machine.”
Misinformation is spreading at an alarming rate. Some notable examples from the past year include: Bot network On X targets the US federal election and President Joe Biden’s deep voicemail encouraging New Hampshire residents to vote. But American voters and tech-savvy people aren’t the targets of most of this content; According to True Media.org analysis, so we tend to underestimate its presence elsewhere.
The volume of deepfakes generated by artificial intelligence increased by 900% between 2019 and 2020, According to According to data from the World Economic Forum.
Meanwhile, there are relatively few deepfakes targeting laws on the books. Detecting deepfakes is set to become a never-ending arms race. Certainly, some tools will not choose to use safety measures such as watermarks, or will be deployed with malicious applications in mind.
Short of radical change, I think the best we can do is be deeply skeptical about what is out there — especially viral content. It’s not as easy as it used to be to distinguish fact from fiction online. But we still have control over what we share versus what we don’t share. This is more impactful than it may seem.
news
ChatGPT research review: My colleague Max took ChatGPT’s new OpenAI search integration, ChatGPT Search, for a spin. He found it impressive in some ways, but unreliable for short queries containing just a few words.
Amazon drones in Phoenix: Just months after ending its drone-based delivery program, Prime Air, California, says it has begun making deliveries to select customers via drone in Phoenix, Arizona.
Former Meta AR Leader Joins OpenAI: The former head of Meta’s augmented reality headset efforts, including Orion, announced Monday that she will join OpenAI to lead robotics and consumer devices. This news comes after OpenAI hired the co-founder of X (formerly Twitter) competitor Pebble.
Has been suspended by account: On Reddit AMASam Altman, CEO of OpenAI, admitted that the lack of computing power is one of the main factors preventing the company from shipping products as much as it would like.
AI-generated summaries: Amazon has launched “X-Ray Recaps,” an AI-powered generative feature that creates concise summaries of entire TV seasons, individual episodes, and even segments of episodes.
Human heights haiku quotes: Anthropic’s latest AI model has arrived: Claude 3.5 Haiku. But it’s more expensive than the last generation, and unlike Anthropic’s other models, it can’t analyze images, charts, or graphs yet.
Apple acquires Pixelmator: AI-powered photo editor Pixelmator Announce On Friday it was acquired by Apple. This deal comes at a time when Apple is becoming more aggressive about integrating artificial intelligence into its photography applications.
Alexa “agent”: Amazon CEO Andy Jassy last week hinted at an improved “agent” version of the company’s Alexa assistant — one that can take actions on a user’s behalf. The revamped Alexa has reportedly faced delays and technical setbacks, and may not launch until sometime in 2025.
Research paper of the week
Web pop-ups can fool AI too, not just grandparents.
In new paperresearchers from Georgia Tech, the University of Hong Kong, and Stanford have shown that AI “agents” — AI models that can complete tasks — can be hijacked by “hostile pop-ups” that direct the models to do things like download malicious file extensions.
It’s pretty clear that some of these pop-ups are traps for the human eye, but the AI isn’t that discerning. The researchers say that the image and text analysis models they tested failed to ignore pop-ups 86% of the time and, as a result, were 47% less likely to complete tasks.
Basic defenses, such as instructing forms to ignore pop-ups, were ineffective. “Deploying PC-based agents still suffers from significant risks, and more robust agent systems are needed to ensure secure agent workflows,” the study co-authors wrote.
Model of the week
Meta announced yesterday that it is working with partners to make Llama’s “open” AI models available for defense applications. Today, one of those partners, Scale AI, announced Lama defensea model built on top of Meta’s Llama 3 that has been “customized and fine-tuned to support U.S. national security missions.”
Llama Defense, available at Scale’s Donavan The chatbot platform, for US government customers, is optimized for planning military and intelligence operations, Scale says. Defense Llama can answer defense-related questions, for example, such as how an adversary might plan an attack against a US military base.
So what makes defensive llamas different from stock llamas? Well, Scale says it’s fine-tuned for content that might be relevant to military operations, such as military doctrine and international humanitarian law, as well as the capabilities of various weapons and defense systems. It’s also not limited to answering war-related questions, as a civilian chatbot might be:
However, it is not clear who might be tempted to use it.
It was the US Army Slow to adopt generative AI – And skeptical about the return on investment. So far, the US Army is only A branch of the US Armed Forces with the spread of artificial intelligence. Military officials have expressed concerns about security vulnerabilities in commercial models, as well as legal challenges associated with sharing intelligence data and the unpredictability of the models when faced with evolving situations.
Grab a bag
Spawning AI, a startup that creates tools to enable creators to opt out of generative AI training, has released an image dataset for training AI models that it claims is entirely public domain.
Most creative AI models are trained on public web data, some of which may be copyrighted or under a restricted license. OpenAI and many other AI vendors argue so Fair use The doctrine protects them from copyright claims. But this did not stop data subjects from Filing lawsuits.
Spawning AI says its training dataset of 12.4 million pairs of image annotations includes only content of “known provenance” and “clear and unambiguous rights labeling” for AI training. Unlike some other datasets, it is also available for download from a dedicated host, eliminating the need to scrape it from the web.
“It is worth noting that the public domain status of the dataset is integral to these larger goals,” Spawning wrote in a blog post. “Data sets that include copyrighted images will continue to rely on web scraping because hosting the images may infringe copyright.”
A Spawning dataset, PD12M, and a formatted version for “aesthetically pleasing” images, PD3M, can be found. On this link.
[ad_2]