{"name":"India AI Safety Network: AI Incident Register","url":"https://indiaaisafety.org/incidents","method":"https://indiaaisafety.org/incidents/method","licence":"https://creativecommons.org/licenses/by/4.0/","updated":"2026-10-01","count":16,"records":[{"id":"2026-003","url":"https://indiaaisafety.org/incidents/2026-003","kind":"incident","title":"Delhi Police face recognition lists people who were in jail as present at a protest","occurred":"2026-07","place":"New Delhi","sector":"Courts, tribunals and policing","system":"Face or voice recognition","system_detail":"Delhi Police’s Facial Recognition System, run from surveillance vans and matched against its biometric database, Crime Kundli, and criminal records.","involvement":"Failed or made an error","harms":["Human or fundamental rights"],"affected":"People named in a police list of 2,873 protesters said to have criminal records","summary":"Delhi Police told the Supreme Court, in an affidavit dated 17 August 2026, that its Facial Recognition System had identified 2,873 people with criminal records at protests at Jantar Mantar between 20 and 26 July. It said 2,402 of the matches came from its biometric database, Crime Kundli, and 471 from criminal records.\n\nThe Indian Express checked 205 people on the list who were linked to serious offences. Police, jail and court records showed that at least 25 of them were in Delhi prisons at the time the system placed them at the protest.","aftermath":"On 1 September 2026 the Supreme Court quashed the police cases against students over the protests, but allowed the government to proceed against the 2,873 people on the list. The police say nobody will face action on the strength of a face match alone, and that each identification will be checked in the field first.","evidence":"Independent reporting","note":"We read the Indian Express findings as reported by MediaNama and Outlook, not in the original. Only 205 of the 2,873 names were checked, and Delhi Police has not published the system’s error rates.","sources":[{"title":"Delhi Police FRS flagged 25 jailed people at Jantar Mantar","publisher":"MediaNama","date":"2026-09-04","url":"https://www.medianama.com/2026/09/223-delhi-polices-facial-recognition-jantar-mantar/","primary":false},{"title":"Jantar Mantar Protest: 25 People Flagged By Facial Recognition Were In Jail, Records Show","publisher":"Outlook India","date":"2026-09-04","url":"https://www.outlookindia.com/national/jantar-mantar-protest-25-people-flagged-by-facial-recognition-were-in-jail-records-show","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2026-002","url":"https://indiaaisafety.org/incidents/2026-002","kind":"incident","title":"Deepfake eye-blink videos allegedly used to pass Aadhaar face checks and take loans in other people’s names","occurred":"2026-04","place":"Ahmedabad, Gujarat","sector":"Finance and payments","system":"Synthetic media","system_detail":"Videos generated from victims’ photographs to get past liveness detection in face authentication. Police said the group used AI tools including Gemini and Meta AI.","involvement":"Used deliberately to cause harm","harms":["Economic or property","Human or fundamental rights"],"affected":"People and businesses whose Aadhaar-linked mobile numbers were changed without their knowledge","summary":"Ahmedabad’s cyber crime police arrested four people in late April 2026 after a businessman complained that the mobile number linked to his Aadhaar had been changed without his consent. Police said the group made deepfake videos with realistic eye blinks from victims’ photographs and showed them during verification to get past liveness checks, and so changed the numbers linked to victims’ Aadhaar records.\n\nWith the new numbers they could receive one-time passwords, get into DigiLocker and banking services, open bank accounts and take personal loans in the victims’ names, according to police.","aftermath":"By 9 May 2026 seven people had been arrested, including two from Assam. Police estimated that the group made between ₹10 lakh and ₹15 lakh a year from fraudulent loans.","evidence":"An official statement","note":"Everything known about the method comes from the police, and none of it has been tested in court. The coverage we read carried no response from UIDAI or from the banks and lenders named.","sources":[{"title":"Four arrested in Gujarat for using AI deepfakes to commit Aadhaar and banking fraud","publisher":"Madhyamam","date":"2026-04-29","url":"https://madhyamamonline.com/amp/india/four-arrested-in-gujarat-for-using-ai-deepfakes-to-commit-aadhaar-and-banking-fraud-1515605","primary":false},{"title":"Assam duo held in AI deepfake Aadhaar fraud linked to interstate loan racket","publisher":"The Assam Tribune (IANS)","date":"2026-05-09","url":"https://assamtribune.com/assam/assam-duo-held-in-ai-deepfake-aadhaar-fraud-linked-to-interstate-loan-racket-1611428","primary":false}],"ai_incident_database":"https://incidentdatabase.ai/cite/1472/","added":"2026-10-01","updated":"2026-10-01"},{"id":"2026-001","url":"https://indiaaisafety.org/incidents/2026-001","kind":"incident","title":"Grok used to make sexualised images of women from their photos, prompting a government notice to X","occurred":"2026-01","place":"Online, across India","sector":"Online platforms and media","system":"Synthetic media","system_detail":"Grok, the AI assistant built by xAI and available inside X, which edited photos of real people on request.","involvement":"Used deliberately to cause harm","harms":["Psychological","Reputational","Human or fundamental rights"],"affected":"Women whose photographs were altered without their consent","summary":"In late December 2025 and early January 2026, users on X repeatedly prompted Grok, the platform’s AI assistant, to produce sexualised versions of photographs of real women, including by removing clothing. The images were generated in public replies and often stayed visible for long periods, MediaNama reported.\n\nOn 2 January 2026 the Ministry of Electronics and Information Technology wrote to X’s chief compliance officer for India. It said Grok was being misused to generate and share obscene images of women, that compliance with the IT Act and the IT Rules, 2021 was not optional, and that X should send an action-taken report within 72 hours.","aftermath":"On 11 January 2026 Akashvani News, citing government sources, reported that X had admitted its mistake and acknowledged gaps in its content moderation, that around 3,500 pieces of content had been blocked and more than 600 accounts deleted, and that X had said it would not allow obscene imagery on the platform.\n\nOn 14 January X said it had put technical measures in place to stop Grok editing images of real people into revealing clothing, and that it would block the generation of such images in places where it is illegal.","evidence":"An official statement","note":"X’s admission is known from government sources quoted by the public broadcaster. The number of people affected in India has not been reported.","sources":[{"title":"Centre issues notice to social media platform ‘X’ over obscene content targeting women, children on AI tool Grok","publisher":"Akashvani News","date":"2026-01-03","url":"https://www.newsonair.gov.in/centre-issues-notice-to-social-media-platform-x-over-obscene-content-targeting-women-children-on-ai-tool-grok","primary":false},{"title":"MeitY Sends Notice To X Over Grok Non-Consensually Editing, Removing Clothing, From Pictures","publisher":"MediaNama","date":"2026-01-03","url":"https://www.medianama.com/2026/01/223-india-directs-x-curb-ai-generated-sexual-content/","primary":false},{"title":"Platform X Admits Lapses in Content Moderation, Assures Compliance with Indian Laws","publisher":"Akashvani News","date":"2026-01-11","url":"https://www.newsonair.gov.in/platform-x-admits-lapses-in-content-moderation-assures-compliance-with-indian-laws","primary":false},{"title":"Musk’s Grok to bar users from generating sexual images of real people","publisher":"Al Jazeera","date":"2026-01-15","url":"https://www.aljazeera.com/news/2026/1/15/musks-grok-to-bar-users-from-generating-sexual-images-of-real-people","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2025-003","url":"https://indiaaisafety.org/incidents/2025-003","kind":"incident","title":"₹425 crore customs penalty set aside because the order rested on fabricated case citations","occurred":"2025-10-08","place":"Surat, Gujarat","sector":"Courts, tribunals and policing","system":"Language model or chatbot","system_detail":"A generative AI tool that the reports do not name. That the officer used one is the Supreme Court’s reading of the fabricated citations.","involvement":"Failed or made an error","harms":["Economic or property","Human or fundamental rights"],"affected":"A diamond exporter penalised on the strength of the order","summary":"On 8 October 2025 the Additional Commissioner of Customs in Surat imposed a penalty of about ₹425.28 crore on a diamond exporter for allegedly declaring natural diamonds as lab-grown. The Gujarat High Court dismissed the exporter’s challenge on 20 January 2026.\n\nOn appeal, a Supreme Court bench of Justices Dipankar Datta and Sheel Nagu found that the customs order relied on judgments that did not exist or carried fake citations, and that some real judgments did not say what the order claimed they said. The bench attributed this to AI hallucination.","aftermath":"In September 2026 the Supreme Court set aside both the penalty order and the High Court’s ruling. It sent the case back to be decided afresh by a different customs officer of the same rank, and left it open to the department to consider disciplinary action against the officer who wrote the original order.","evidence":"A court or regulator’s finding","note":"","sources":[{"title":"Supreme Court scraps ₹425 crore penalty over fake citations, AI hallucinations in customs order","publisher":"Bar and Bench","date":"2026-09-02","url":"https://www.barandbench.com/news/litigation/supreme-court-scraps-425-crore-penalty-over-fake-citations-ai-hallucinations-in-customs-order","primary":false},{"title":"Supreme Court overturns Rs 425 crore customs penalty over AI-generated fake citations","publisher":"MediaNama","date":"2026-09-03","url":"https://www.medianama.com/2026/09/223-supreme-court-ai-hallucination-customs-order/","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2025-001","url":"https://indiaaisafety.org/incidents/2025-001","kind":"incident","title":"Mandatory face authentication blocks take-home rations for women and children at anganwadis","occurred":"2025-07-01","place":"Across India","sector":"Welfare and public services","system":"Face or voice recognition","system_detail":"Face authentication in the Poshan Tracker app, which is owned by the Ministry of Women and Child Development and built by the National e-Governance Division. A live photo is compared with the person’s Aadhaar photograph.","involvement":"Worked as built, and the way it was used caused harm","harms":["Human or fundamental rights"],"affected":"Pregnant and breastfeeding women and young children entitled to take-home rations, and the anganwadi workers who run the checks","summary":"From 1 July 2025 the Ministry of Women and Child Development made face recognition mandatory for pregnant and breastfeeding women collecting take-home rations under Poshan Abhiyaan. Anganwadi workers verify each woman in the Poshan Tracker app, which compares a live photo with her Aadhaar photograph. From the first weeks, workers reported failed face scans, one-time passwords that never arrived and server crashes.\n\nA Decode investigation in Bihar, Jharkhand and Karnataka, published in March 2026, found children cut off from meals because the system could not match their mothers’ faces. One worker said she had been told to take at least four children off her centre’s rolls. Government data cited in the report show that 52.7 percent of eligible beneficiaries received rations by the end of 2025.","aftermath":"Anganwadi workers’ federations asked for the rule to be withdrawn, and a union in Karnataka challenged it in the High Court, which on 23 April 2026 asked the state government for its position. The ministry says face authentication stops rations being diverted. It did not answer Decode’s questions about whether the system has been independently audited.","evidence":"Independent reporting","note":"Many of the reported failures come from poor connectivity and one-time passwords, not from the face match itself. There is no official count of people who were excluded.","sources":[{"title":"Mandatory facial recognition for women’s nutrition scheme sparks fears of exclusion and tech failures","publisher":"Down To Earth","date":"2025-07-01","url":"https://www.downtoearth.org.in/food/mandatory-facial-recognition-for-womens-nutrition-scheme-sparks-fears-of-exclusion-and-tech-failures","primary":false},{"title":"India’s Facial Recognition Drive On Hungry Children Is Erasing Them","publisher":"Decode","date":"2026-03-25","url":"https://www.decodeinternet.in/decode/indias-facial-recognition-drive-on-hungry-children-is-erasing-them-30906","primary":false},{"title":"Karnataka High Court Seeks State’s Stand On Anganwadi Workers’ Plea Against Mandatory Facial Recognition To Avail Take-Home Ration Scheme","publisher":"LiveLaw","date":"2026-04-24","url":"https://www.livelaw.in/high-court/karnataka-high-court/karnataka-high-court-anganwadi-workers-mandatory-facial-recognition-take-home-ration-poshan-20-531594","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2025-002","url":"https://indiaaisafety.org/incidents/2025-002","kind":"incident","title":"AI-generated sexual images of a woman posted under a fake persona that drew over a million followers","occurred":"2025-07","place":"Assam","sector":"Online platforms and media","system":"Synthetic media","system_detail":"Image-generation tools. Police named different ones to different outlets, among them ChatGPT, Dzine, Midjourney and OpenArt.","involvement":"Used deliberately to cause harm","harms":["Psychological","Reputational","Human or fundamental rights"],"affected":"A woman whose photographs were used without her knowledge","summary":"An Instagram account under an invented name had 1.4 million followers by July 2025, with pictures and videos of what appeared to be a young Assamese woman. The face was that of a real woman in Dibrugarh, who is not on social media and learned about the account only when it went viral. Police said a man she knew had built the persona from private photographs of her, at first by altering real pictures and later by generating new photos and videos with AI tools.\n\nPolice believe he made about ₹10 lakh by selling access to explicit content through paid subscriptions.","aftermath":"Her family complained to the police in Dibrugarh on 11 July 2025 and the man was arrested the next day. Police said the motive was revenge, and seized his devices and bank records. The account is no longer public, but the BBC reported that its images were still circulating on social media.","evidence":"An official statement","note":"Reports differ on which AI tools were used and on how the two knew each other, and the allegations have not been tested in court. We have left out the account’s name because it echoes the woman’s own.","sources":[{"title":"Deepfake deception: Indian woman’s identity stolen for erotic AI content","publisher":"BBC News","date":"2025-07-23","url":"https://www.bbc.com/news/articles/cn0znk47x9eo","primary":false},{"title":"Assam engineer held for AI-generated porn scandal targeting former classmate","publisher":"Gulf News","date":"2025-07-14","url":"https://gulfnews.com/world/asia/india/assam-engineer-held-for-ai-generated-porn-scandal-targeting-former-classmate-1.500197524","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2024-005","url":"https://indiaaisafety.org/incidents/2024-005","kind":"incident","title":"Income tax tribunal recalls an order that cited four judgments which do not exist","occurred":"2024-12","place":"Bengaluru, Karnataka","sector":"Courts, tribunals and policing","system":"Language model or chatbot","system_detail":"ChatGPT, according to press reports that the tax department’s representatives had used it. The tribunal itself did not say so.","involvement":"Failed or made an error","harms":["Human or fundamental rights"],"affected":"The taxpayer in the case, a trust in a dispute reported to be worth ₹669 crore","summary":"In December 2024 the Bengaluru bench of the Income Tax Appellate Tribunal ruled for the tax department in Buckeye Trust v PCIT, a dispute over the transfer of a partnership interest to a trust, reported to be worth ₹669 crore. The order relied on three Supreme Court judgments and one Madras High Court judgment that were later found not to exist.\n\nPress reports said the citations had been generated by ChatGPT, which representatives of the tax department were reported to have used to prepare their case.","aftermath":"The tribunal recalled the order about a week later, and the case was listed to be heard again.","evidence":"Independent reporting","note":"That ChatGPT was the source rests on press reports, not on anything the tribunal has said.","sources":[{"title":"Bengaluru Tax Tribunal Withdraws Order Based on Non-Existent Case Law Cited by ChatGPT","publisher":"Law Trend","date":"2025-02-28","url":"https://lawtrend.in/bengaluru-tax-tribunal-withdraws-order-based-on-non-existent-case-law-cited-by-chatgpt/","primary":false},{"title":"10 cases that show Indian courts have an AI hallucination problem","publisher":"MediaNama","date":"2026-07-03","url":"https://www.medianama.com/2026/07/223-10-cases-ai-hallucination-cases-in-indian-courts/","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2024-004","url":"https://indiaaisafety.org/incidents/2024-004","kind":"incident","title":"Deepfake videos of the RBI Governor used to promote investment schemes","occurred":"2024-11","place":"Across India","sector":"Finance and payments","system":"Synthetic media","system_detail":"Fake videos of the Governor. The Reserve Bank did not say how they were made or by whom.","involvement":"Used deliberately to cause harm","harms":["Public interest","Reputational"],"affected":"Members of the public targeted with fake investment advice, and the Reserve Bank","summary":"On 19 November 2024 the Reserve Bank of India said that fake videos of its Governor were circulating on social media, claiming that the RBI had launched or was backing certain investment schemes and advising people to put money into them.\n\nThe RBI said its officials had nothing to do with the videos, that they were fake and that it never gives investment advice, and it warned people not to fall for what it called deepfake videos. Its statement did not say who made them, or whether anyone had lost money.","aftermath":"The warning followed a similar one from the National Stock Exchange in April 2024, about deepfake videos of its chief executive recommending stocks.","evidence":"An official statement","note":"","sources":[{"title":"Fake investment advice: RBI warns investors on expert deepfake videos circulated over social media","publisher":"Business Today","date":"2024-11-19","url":"https://www.businesstoday.in/personal-finance/investment/story/fake-investment-advice-rbi-warns-investors-on-expert-deepfake-videos-circulated-over-social-media-check-details-454294-2024-11-19","primary":false},{"title":"RBI warns people over deepfake videos of Governor","publisher":"The Tribune","date":"2024-11-20","url":"https://www.tribuneindia.com/news/business/rbi-warns-people-over-deepfake-videos-of-governor/amp","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2024-003","url":"https://indiaaisafety.org/incidents/2024-003","kind":"incident","title":"AI voice clones of two Bollywood actors used in fake election videos","occurred":"2024-04","place":"Across India","sector":"Elections and public information","system":"Synthetic media","system_detail":"Real footage of the actors with AI-cloned voices laid over it.","involvement":"Used deliberately to cause harm","harms":["Public interest","Reputational"],"affected":"Voters shown fabricated endorsements during the general election, and the two actors","summary":"During the 2024 general election, two videos went viral in which the actors Aamir Khan and Ranveer Singh appeared to criticise the Prime Minister and ask people to vote for the Congress party. Reuters counted more than half a million views within a week and could not establish who had made them.\n\nThe fact-checking outlet BOOM found that each video used real footage with an AI-cloned voice laid over it. The Aamir Khan clip came from a 2014 promotion for his television show, and the Ranveer Singh clip from a recent visit to Varanasi.","aftermath":"Aamir Khan’s office said he had never endorsed a political party, and he registered a case with Mumbai Police against unnamed persons. Ranveer Singh also complained, and the police registered a case against the account that had promoted the video. Facebook told Reuters it had removed the videos.","evidence":"Independent reporting","note":"","sources":[{"title":"Congress Functionaries Share AI Voice Clone Of Aamir Khan Targeting PM Modi","publisher":"BOOM","date":"2024-04-16","url":"https://www.boomlive.in/fact-check/aamir-khan-jhumla-congress-bjp-old-video-fact-check-24920","primary":false},{"title":"Video Of Ranveer Singh Criticising PM Modi Is A Deepfake AI Voice Clone","publisher":"BOOM","date":"2024-04-18","url":"https://www.boomlive.in/fact-check/viral-video-bollywood-actor-ranveer-singh-congress-campaign-lok-sabha-elections-claim-social-media-24940","primary":false},{"title":"Aamir Khan lodges FIR for his alleged deep fake video used in poll advertisement","publisher":"The Tribune (IANS)","date":"2024-04-16","url":"https://www.tribuneindia.com/news/entertainment/aamir-khan-lodges-fir-against-congress-for-alleged-deep-fake-video-used-in-poll-advertisement-611383/","primary":false},{"title":"Mumbai Police registers case over deepfake video of Ranveer Singh","publisher":"ANI","date":"2024-04-23","url":"https://www.aninews.in/news/entertainment/bollywood/mumbai-police-registers-case-over-deepfake-video-of-ranveer-singh20240423235122","primary":false},{"title":"Deepfakes of Bollywood stars spark worries of AI meddling in India election","publisher":"Reuters, republished by Rappler","date":"2024-04-22","url":"https://www.rappler.com/technology/internet-culture/deepfakes-bollywood-stars-spark-worries-ai-meddling-india-election/","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2024-002","url":"https://indiaaisafety.org/incidents/2024-002","kind":"incident","title":"Deepfake videos of the National Stock Exchange’s chief executive give stock tips","occurred":"2024-04","place":"Across India","sector":"Finance and payments","system":"Synthetic media","system_detail":"Audio and video clips imitating the chief executive’s voice and facial expressions, carrying the exchange’s logo.","involvement":"Used deliberately to cause harm","harms":["Public interest","Reputational"],"affected":"Investors shown fake stock recommendations, and the exchange","summary":"On 10 April 2024 the National Stock Exchange warned investors about videos and audio clips in which its chief executive appeared to recommend stocks. The exchange said the clips had been made with technology that imitated his voice and facial expressions, and that they carried its logo.\n\nIt told investors not to act on them, said its employees are not allowed to recommend stocks, and said it was asking platforms to take the clips down.","aftermath":"In July 2024 the Bombay High Court directed social media platforms to act against deepfake videos of the exchange’s chief executive, Business Today reported. In November 2024 the Reserve Bank of India issued a similar warning about deepfake videos of its Governor.","evidence":"An official statement","note":"","sources":[{"title":"Beware of deepfake of CEO recommending stocks, says National Stock Exchange","publisher":"Business Standard","date":"2024-04-10","url":"https://www.business-standard.com/markets/news/beware-of-deepfake-of-ceo-recommending-stocks-says-national-stock-exchange-124041000420_1.html","primary":false},{"title":"National Stock Exchange cautions against deepfake videos of its chief on stocks","publisher":"The Tribune","date":"2024-04-11","url":"https://www.tribuneindia.com/news/business/national-stock-exchange-cautions-against-deepfake-videos-of-its-chief-on-stocks-609608/","primary":false},{"title":"Fake investment advice: RBI warns investors on expert deepfake videos circulated over social media","publisher":"Business Today","date":"2024-11-19","url":"https://www.businesstoday.in/personal-finance/investment/story/fake-investment-advice-rbi-warns-investors-on-expert-deepfake-videos-circulated-over-social-media-check-details-454294-2024-11-19","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2024-001","url":"https://indiaaisafety.org/incidents/2024-001","kind":"incident","title":"Deepfake of Sachin Tendulkar used to advertise a gaming app","occurred":"2024-01","place":"Online","sector":"Online platforms and media","system":"Synthetic media","system_detail":"A manipulated clip with an imitation of his voice, shared on Facebook.","involvement":"Used deliberately to cause harm","harms":["Reputational","Public interest"],"affected":"The cricketer, and viewers shown a false endorsement","summary":"In January 2024 a video shared on Facebook appeared to show Sachin Tendulkar promoting an online gaming app and saying his daughter had made money from it. The clip and the voice had been digitally manipulated.\n\nTendulkar posted the video himself on 15 January, said it was fake and asked people to report such videos, ads and apps.","aftermath":"Mumbai Police registered a case against a gaming site and a Facebook page on a complaint from his office, under the defamation section of the penal code and the IT Act. The minister of state for IT said deepfakes were a threat to the safety and trust of Indian users, and the government promised tighter rules.","evidence":"An official statement","note":"","sources":[{"title":"Govt promises tighter norms after Sachin Tendulkar flags deepfake video","publisher":"The Tribune","date":"2024-01-16","url":"https://www.tribuneindia.com/news/india/govt-promises-tighter-norms-after-sachin-flags-deepfake-video-581609/","primary":false},{"title":"FIR against gaming site, FB page as Tendulkar raises concerns over his deepfake video","publisher":"The Tribune (PTI)","date":"2024-01-18","url":"https://www.tribuneindia.com/news/india/fir-against-gaming-site-fb-page-as-tendulkar-raises-concerns-over-his-deepfake-video-582400/","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2023-002","url":"https://indiaaisafety.org/incidents/2023-002","kind":"incident","title":"Deepfake video of the actor Rashmika Mandanna spreads on social media","occurred":"2023-11","place":"Online","sector":"Online platforms and media","system":"Synthetic media","system_detail":"An AI-generated deepfake video. The tool used has not been named.","involvement":"Used deliberately to cause harm","harms":["Reputational","Psychological"],"affected":"The actor whose face was used, and the woman whose video was altered","summary":"In early November 2023 a video that appeared to show the actor Rashmika Mandanna spread widely on social media. Police later said it had been made by putting her face onto a video that another woman had posted on Instagram. It drew public concern from the Prime Minister and from other actors.\n\nOn 10 November Delhi Police registered a case of forgery and identity theft on a complaint from the Delhi Commission for Women, and wrote to Meta for details of the account that had posted the video.","aftermath":"On 7 November 2023 the IT ministry sent an advisory to large social media platforms, reminding them that they must remove such content within 36 hours of a complaint. On 20 January 2024 Delhi Police said they had arrested a man from Guntur in Andhra Pradesh who, they said, made the video to gain followers for a fan page he ran.","evidence":"An official statement","note":"The ministry’s advisory does not name this case; we link the two by their timing. The allegations against the man arrested have not been tested in court.","sources":[{"title":"Union Government issues advisory to social media intermediaries to identify misinformation and deepfakes","publisher":"Press Information Bureau","date":"2023-11-07","url":"https://www.pib.gov.in/PressReleasePage.aspx?PRID=1975445","primary":true},{"title":"Rashmika Mandanna deep fake case: Delhi Police arrests mastermind from Andhra Pradesh","publisher":"ANI","date":"2024-01-20","url":"https://www.aninews.in/news/national/general-news/rashmika-mandanna-deep-fake-case-delhi-police-arrests-mastermind-from-andhra-pradesh20240120175512","primary":false},{"title":"Creator of deepfake video of actress Rashmika Mandanna arrested from Andhra’s Guntur; wanted to increase Instagram followers: Delhi Police","publisher":"The Tribune (PTI)","date":"2024-01-20","url":"https://www.tribuneindia.com/news/india/creator-of-deepfake-video-of-actor-rashmika-mandanna-arrested-from-south-india-583046/","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2023-001","url":"https://indiaaisafety.org/incidents/2023-001","kind":"incident","title":"Deepfake video call impersonating a former colleague used to take ₹40,000","occurred":"2023-07-09","place":"Kozhikode, Kerala","sector":"Finance and payments","system":"Synthetic media","system_detail":"A face and voice faked on a WhatsApp video call. Police described an AI-based video tool that worked with a blurred background and simple movements of the chin, eyes and lips.","involvement":"Used deliberately to cause harm","harms":["Economic or property"],"affected":"A retired man in his seventies","summary":"On 9 July 2023 a retired man in Kozhikode got a WhatsApp video call from someone who looked and sounded like a former colleague. The caller mentioned other people they had worked with, said a relative was in hospital and asked for ₹40,000, which he sent online.\n\nWhen the caller asked for another ₹35,000 he became suspicious, checked with other former colleagues and found that the call was not genuine. Kerala Police’s cyber wing said the fraudsters had faked the colleague’s face and voice.","aftermath":"Police traced the money through a Jio Payments Bank account to an account at a bank branch in Maharashtra and asked the bank to freeze it. By January 2024 four people had been arrested in the case. One of them, already held in Delhi’s Tihar Jail in a similar case, was brought before a court in Kozhikode for questioning.","evidence":"An official statement","note":"The account of the technology comes from the police. No forensic analysis of the call has been published.","sources":[{"title":"Coal India’s retd official loses Rs 40K after he gets deep-fake video call from ex-colleague asking for money","publisher":"Onmanorama","date":"2023-07-17","url":"https://www.onmanorama.com/news/kerala/2023/07/17/ai-based-deep-fake-man-conned-kozhikode-account-owner.html","primary":false},{"title":"WhatsApp video call scam: Kerala man robbed of Rs 40,000 by stranger pretending to be former colleague","publisher":"Business Today","date":"2023-07-17","url":"https://www.businesstoday.in/technology/news/story/whatsapp-video-call-scam-kerala-man-robbed-of-rs-40000-by-stranger-pretending-to-be-former-colleague-390035-2023-07-17","primary":false},{"title":"Deepfake scam: Ahmedabad-based accused questioned in Kozhikode","publisher":"Onmanorama","date":"2024-01-17","url":"https://www.onmanorama.com/news/kerala/2024/01/17/man-incarcerated-in-tihar-jail-presented-before-kozhikode-cjm-court.html","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2020-002","url":"https://indiaaisafety.org/incidents/2020-002","kind":"incident","title":"Family ID database in Haryana records living pensioners as dead and stops their pensions","occurred":"2020-06","place":"Haryana","sector":"Welfare and public services","system":"Automated decision system","system_detail":"Parivar Pehchan Patra, the state’s family identity database, run by the Citizen Resources Information Department. Officials have said it uses artificial intelligence and machine learning. How it reaches its decisions has not been published.","involvement":"Failed or made an error","harms":["Economic or property","Human or fundamental rights"],"affected":"Elderly people and widows whose pensions were stopped","summary":"In June 2020 Haryana began using its family identity database, the Parivar Pehchan Patra, to decide who was eligible for welfare. The database recorded some living people as dead and others with incomes they did not have, and their pensions stopped. In September 2022 a 102-year-old man led a mock wedding procession through Rohtak to show officials he was alive.\n\nFigures the government gave the state assembly in August 2023 showed that it had stopped the pensions of 277,115 elderly people and 52,479 widows over three years because they were recorded as dead. According to an investigation published by Al Jazeera, several thousand of them were alive.","aftermath":"After protests the state held grievance camps to review the data. On 29 August 2023 the Chief Minister said that of 63,353 people whose old-age pensions had been halted on the basis of the database, 44,050 were later found to be eligible, and that most of the errors had been corrected.","evidence":"An official statement","note":"The government has not said whether the mistakes came from wrong data entry or from the system’s own inferences. Both sources come from the same reporting team.","sources":[{"title":"In India, an algorithm declares them dead; they have to prove they’re alive","publisher":"Al Jazeera","date":"2024-01-25","url":"https://www.aljazeera.com/economy/2024/1/25/in-india-an-algorithm-declares-them-dead-they-have-to-prove-theyre","primary":false},{"title":"Haryana’s Family ID declared thousands of people dead, halted their pension","publisher":"The Reporters’ Collective","date":"2024-01-25","url":"https://www.reporters-collective.in/newsletters/ai-part-2","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2020-001","url":"https://indiaaisafety.org/incidents/2020-001","kind":"hazard","title":"Delhi Police treats face recognition matches above 80% similarity as positive identifications","occurred":"2020","place":"Delhi","sector":"Courts, tribunals and policing","system":"Face or voice recognition","system_detail":"Delhi Police’s facial recognition system, which compares footage with photographs of convicts and police dossiers.","involvement":"Worked as built, and the way it was used caused harm","harms":["Human or fundamental rights"],"affected":"People identified as suspects from CCTV and video footage","summary":"Delhi Police used facial recognition to identify suspects after the riots in north-east Delhi in February 2020, and later in its investigations of the 2021 farmers’ rally at the Red Fort and the 2022 Jahangirpuri riots. The police commissioner at the time of the 2020 riots said that 231 people were arrested on the basis of CCTV or video footage, and that 137 of them were identified through facial recognition.\n\nIn replies dated 25 July 2022 to right-to-information requests from the Internet Freedom Foundation, the police said they treat any match above 80 percent similarity as a positive result, and keep weaker matches as leads that need other evidence. They also said that no privacy impact assessment had been carried out.","aftermath":"The police answered only after the Central Information Commission directed them, in June 2022, to respond to the requests afresh.","evidence":"An official statement","note":"This record is a hazard: a practice that could plausibly lead to people being wrongly identified. We found no court finding that a particular person was misidentified in these investigations. Delhi Police denies that its use of the technology is biased.","sources":[{"title":"Delhi Police’s claims that FRT is accurate with a 80% match are 100% scary","publisher":"Internet Freedom Foundation","date":"2022-08-17","url":"https://internetfreedom.in/delhi-polices-frt-use-is-80-accurate-and-100-scary/","primary":false},{"title":"Indian police use facial recognition to persecute Muslims and other marginalized communities","publisher":"Coda Story","date":"2022-10-11","url":"https://www.codastory.com/surveillance-and-control/india-police-facial-recognition/","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"},{"id":"2016-001","url":"https://indiaaisafety.org/incidents/2016-001","kind":"incident","title":"Welfare algorithm in Telangana wrongly tags poor families as ineligible for subsidised food","occurred":"2016","place":"Telangana","sector":"Welfare and public services","system":"Automated decision system","system_detail":"Samagra Vedika, built by Posidex Technologies and run by the state government. It links a person’s records across government databases, using what the state calls machine learning and entity resolution.","involvement":"Failed or made an error","harms":["Economic or property","Human or fundamental rights"],"affected":"Families whose food security cards were cancelled or refused","summary":"From 2016 Telangana used a system called Samagra Vedika to check who was eligible for welfare. It pulls together a person’s records from several government databases and flags signs of ineligibility, such as owning a car. An investigation published by Al Jazeera in January 2024 found that it had matched people to assets they did not own. In one case a widow lost her food security card because the system confused her late husband, a rickshaw puller, with a car owner of a similar name.\n\nBetween 2014 and 2019 the state cancelled more than 1.86 million food security cards and rejected 142,086 new applications without notice. Not all of these can be put down to the system. But when the Supreme Court ordered the cancelled cards to be checked again in the field, 15,471 of the first 205,734 applications examined were approved, which suggests that at least 7.5 percent had been wrongly rejected.","aftermath":"In April 2022 the Supreme Court directed Telangana to carry out field verification of all the cancelled cards and to deal with representations from anyone affected. The state’s IT department refused requests under the Right to Information Act for the system’s source code and data formats, saying the company held rights over them.","evidence":"Independent reporting","note":"The Telangana government disputes the findings. Its IT department told Al Jazeera that checks by several departments had found very high levels of accuracy.","sources":[{"title":"How an algorithm denied food to thousands of poor in India’s Telangana","publisher":"Al Jazeera","date":"2024-01-24","url":"https://www.aljazeera.com/economy/2024/1/24/how-an-algorithm-denied-food-to-thousands-of-poor-in-indias-telangana","primary":false},{"title":"‘You Cannot Cancel Before Verification’: Supreme Court On Telangana Govt Cancelling 19 Lakh Ration Cards; Directs Field Verification","publisher":"LiveLaw","date":"2022-04-27","url":"https://www.livelaw.in/top-stories/supreme-court-ration-cards-cancellation-telangana-govt-conduct-field-verification-197664","primary":false}],"ai_incident_database":"","added":"2026-10-01","updated":"2026-10-01"}]}