Posted by Dave Kleidermacher, VP, Head of Security & Privacy - Android & Play
Providing users with safe and secure experiences, while helping developers build and grow quality app businesses, is our top priority at Google Play. And we’re constantly working to improve our protections.
Google Play has been working to minimize app install attribution fraud for several years. In 2017 Google Play made available the Google Play Install Referrer API, which allows ad attribution providers, publishers and advertisers to determine which referrer was responsible for sending the user to Google Play for a given app install. This API was specifically designed to be resistant to install attribution fraud and we strongly encourage attribution providers, advertisers and publishers to insist on this standard of proof when measuring app install ads. Users, developers, advertisers and ad networks all benefit from a transparent, fair system.
We also take reports of questionable activity very seriously. If an app violates our Google Play Developer policies, we take action. That’s why we began our own independent investigation after we received reports of apps on Google Play accused of conducting app install attribution abuse by falsely claiming credit for newly installed apps to collect the download bounty from that app’s developer.
We now have an update regarding our ongoing investigation:
On Monday, we removed two apps from the Play Store because our investigation discovered evidence of app install attribution abuse.
We also discovered evidence of app install attribution abuse in 3 ad network SDKs. We have asked the impacted developers to remove those SDKs from their apps. Because we believe most of these developers were not aware of the behavior from these third-party SDKs, we have given them a short grace period to take action.
Google Ads SDKs were not utilized for any of the abusive behaviors mentioned above.
Our investigation is ongoing and additional reviews of other apps and third party SDKs are still underway. If we find evidence of additional policy violations, we will take action.
We will continue to investigate and improve our capabilities to better detect and protect against abusive behavior and the malicious actors behind them.
Posted by Billy Lau and René Mayrhofer, Android Security & Privacy Team
Customization is one of Android's greatest strengths. Android's open source nature has enabled thousands of device types that cover a variety of use cases. In addition to adding features to the Android Open Source Project, researchers, developers, service providers, and device and chipset manufacturers can make updates to improve Android security. Investing and engaging in academic research advances the state-of-the-art security techniques, contributes to science, and delivers cutting edge security and privacy features into the hands of end users. To foster more cooperative applied research between the Android Security and Privacy team and the wider academic and industrial community, we're launching ASPIRE (Android Security and PrIvacy REsearch).
ASPIRE's goal is encouraging the development of new security and privacy technology that impacts the Android ecosystem in the next 2 to 5 years, but isn't planned for mainline Android development. This timeframe extends beyond the next annual Android release to allow adequate time to analyze, develop, and stabilize research into features before including in the platform. To collaborate with security researchers, we're hosting events and creating more channels to contribute research.
On October 25th 2018, we invited top security and privacy researchers from around the world to present at Android Security Local Research Day (ASLR-D). At this event, external researchers and Android Security and Privacy team members discussed current issues and strategies that impact the future direction of security research—for Android and the entire industry.
We can't always get everyone in the same room and good ideas come from everywhere. So we're inviting all academic researchers to help us protect billions of users. Research collaborations with Android should be as straightforward as collaborating with the research lab next door. To get involved you can:
Posted by Per Bjorke, Product Manager, Ad Traffic Quality
For years, Google has been waging a comprehensive, global fight against invalid traffic through a combination of technology, policy, and operations teams to protect advertisers and publishers and increase transparency throughout the advertising industry.
Last year, we identified one of the most complex and sophisticated ad fraud operations we have seen to date, working with cyber security firm White Ops, and referred the case to law enforcement. Today, the U.S. Attorney’s Office for the Eastern District of New York announced criminal charges associated with this fraud operation. This takedown marks a major milestone in the industry’s fight against ad fraud, and we’re proud to have been a key contributor.
In partnership with White Ops, we have published a white paper about how we identified this ad fraud operation, the steps we took to protect our clients from being impacted, and the technical work we did to detect patterns across systems in the industry. Below are some of the highlights from the white paper, which you can download here. All about 3ve: A creative and sophisticated threat Referred to as 3ve (pronounced “Eve”), this ad fraud operation evolved over the course of 2017 from a modest, low-level botnet into a large and sophisticated operation that used a broad set of tactics to commit ad fraud. 3ve operated on a significant scale: At its peak, it controlled over 1 million IPs from both residential malware infections and corporate IP spaces primarily in North America and Europe.
Through our investigation, we discovered that 3ve was comprised of three unique sub-operations that evolved rapidly, using sophisticated tactics aimed at exploiting data centers, computers infected with malware, spoofed fraudulent domains, and fake websites. Through its varied and complex machinery, 3ve generated billions of fraudulent ad bid requests (i.e., ad spaces on web pages that advertisers can bid to purchase in an automated way), and it also created thousands of spoofed fraudulent domains. It should be noted that our analysis of ad bid requests indicated growth in activity, but not necessarily growth in transactions that would result in charges to advertisers. It’s also worth noting that 3+ billion daily ad bid requests made 3ve an extremely large ad fraud operation, but its bid request volume was only a small percentage of overall bid request volume across the industry.
Our objective Trust and integrity are critical to the digital advertising ecosystem. Investments in our ad traffic quality systems made it possible for us to tackle this ad fraud operation and to limit the impact it had on our clients as quickly as possible, including crediting advertisers.
3ve’s focus, like many ad fraud schemes, was not a single player or system, but rather the whole advertising ecosystem. As we worked to protect our ad systems against traffic from this threat, we identified that others also had observed this traffic, and we partnered with them to help remove the threat from the ecosystem. The working group, which included nearly 20 partners, was a key component that shaped our broader investigation into 3ve, enabling us to engage directly with each other and to work towards a mutually beneficial outcome.
Industry collaboration helps bring 3ve down
While ad fraud traditionally has been seen as a faceless crime in which bad actors don’t face much risk of being identified or consequences for their actions, 3ve’s takedown demonstrates that there are risks and consequences to committing ad fraud. We’re confident that our collective efforts are building momentum and moving us closer to finding a resolution to this challenge.
For example, industry initiatives such as the Interactive Advertising Bureau (IAB) Tech Lab’s ads.txt standard, which has experienced and continues to see very rapid adoption (over 620,000 domains have an ads.txt), as well as the increasing number of buy-side platforms and exchanges offering refunds for invalid traffic, are valuable steps towards cutting off the money flow to fraudsters. As we announced last year, we’ve made, and will continue to make investments in our automated refunds for invalid traffic, including our work with supply partners to provide advertisers with refunds for invalid traffic detected up to 30 days after monthly billing.
Industry bodies such as the IAB, Trustworthy Accountability Group (TAG), Media Rating Council, and the Joint Industry Committee for Web Standards, who are serving as agents of change and collaboration across our industry, are instrumental in the fight against ad fraud. We have a long history of working with these bodies, including ongoing participation in TAG and IAB leadership and working groups, as well as our inclusion in the TAG Certified Against Fraud program. That program’s value was reinforced with the IAB’s requirement that all members need to be TAG certified by the middle of this year.
A coordinated takedown of infrastructure related to 3ve’s operations occurred recently. The takedown involved disrupting as much of the related infrastructure as possible to make it hard to rebuild any of 3ve’s operations. As the graph below demonstrates, declining volumes in invalid traffic indicate that the disruption thus far has been successful, bringing the bid request traffic close to zero within 18 hours of starting the coordinated takedown.
We’ll continue to be vigilant, working to protect marketers, publishers, and users, while continuing to collaborate with the broader industry to safeguard the integrity of the digital advertising ecosystem that powers the open web. Our work to take down 3ve is another example of our collaboration with the broader ecosystem to improve trust in digital advertising. We are committed to helping to create a better digital advertising ecosystem — one that is more valuable, transparent, and trusted for everyone.
This Android Ecosystem Security Transparency Report covers how often a routine, full-device scan by Google Play Protect detects a device with PHAs installed. Google Play Protect is built-in protection on Android devices that scans over 50 billion apps daily from inside and outside of Google Play. These scans look for evidence of Potentially Harmful Applications (PHAs). If the scans find a PHA, Google Play Protect warns the user and can disable or remove PHAs. In Android's first annual Android Security Year in Review from 2014, fewer than 1% of devices had PHAs installed. The percentage has declined steadily over time and this downward trend continues through 2018. The transparency report covers PHA rates in three areas: market segment (whether a PHA came from Google Play or outside of Google Play), Android version, and country.
Devices with Potentially Harmful Applications installed by market segment Google works hard to protect your Android device: no matter where your apps come from. Continuing the trend from previous years, Android devices that only download apps from Google Play are 9 times less likely to get a PHA than devices that download apps from other sources. Before applications become available in Google Play they undergo an application review to confirm they comply with Google Play policies. Google uses a risk scorer to analyze apps to detect potentially harmful behavior. When Google’s application risk analyzer discovers something suspicious, it flags the app and refers the PHA to a security analyst for manual review if needed. We also scan apps that users download to their device from outside of Google Play. If we find a suspicious app, we also protect users from that—even if it didn't come from Google Play.
In the Android Ecosystem Security Transparency Report, the Devices with Potentially Harmful Applications installed by market segment chart shows the percentage of Android devices that have one or more PHAs installed over time. The chart has two lines: PHA rate for devices that exclusively install from Google Play and PHA rate for devices that also install from outside of Google Play. In 2017, on average 0.09% of devices that exclusively used Google Play had one or more PHAs installed. The first three quarters in 2018 averaged a lower PHA rate of 0.08%.
The security of devices that installed apps from outside of Google Play also improved. In 2017, ~0.82% of devices that installed apps from outside of Google Play were affected by PHA; in the first three quarters of 2018, ~0.76% were affected. Since 2017, we've reduced this number by expanding the auto-disable feature which we covered on page 10 in the 2017 Year in Review. While malware rates fluctuate from quarter to quarter, our metrics continue to show a consistent downward trend over time. We'll share more details in our 2018 Android Security Year in Review in early 2019.
Devices with Potentially Harmful Applications installed by Android version Newer versions of Android are less affected by PHAs. We attribute this to many factors, such as continued platform and API hardening, ongoing security updates and app security and developer training to reduce apps' access to sensitive data. In particular, newer Android versions—such as Nougat, Oreo, and Pie—are more resilient to privilege escalation attacks that had previously allowed PHAs to gain persistence on devices and protect themselves against removal attempts. The Devices with Potentially Harmful Applications installed by Android version chart shows the percentage of devices with a PHA installed, sorted by the Android version that the device is running.
Devices with Potentially Harmful Applications rate by top 10 countries Overall, PHA rates in the ten largest Android markets have remained steady. While these numbers fluctuate on a quarterly basis due to the fluidity of the marketplace, we intend to provide more in depth coverage of what drove these changes in our annual Year in Review in Q1, 2019.
The Devices with Potentially Harmful Applications rate by top 10 countries chart shows the percentage of devices with at least one PHA in the ten countries with the highest volume of Android devices. India saw the most significant decline in PHAs present on devices, with the average rate of infection dropping by 32 percent. Indonesia, Mexico, and Turkey also saw a decline in the likelihood of PHAs being present on devices in the region. South Korea saw the lowest number of devices containing PHA, with only 0.12%.
Posted by Matt Ruhstaller, TPM and Oliver Chang, Software Engineer, Google Security Team
Open Source Software (OSS) is extremely important to Google, and we rely on OSS in a variety of customer-facing and internal projects. We also understand the difficulty and importance of securing the open source ecosystem, and are continuously looking for ways to simplify it.
For the OSS community, we currently provide OSS-Fuzz, a free continuous fuzzing infrastructure hosted on the Google Cloud Platform. OSS-Fuzz uncovers security vulnerabilities and stability issues, and reports them directly to developers. Since launching in December 2016, OSS-Fuzz has reported over 9,000 bugs directly to open source developers.
In addition to OSS-Fuzz, Google's security team maintains several internal tools for identifying bugs in both Google internal and Open Source code. Until recently, these issues were manually reported to various public bug trackers by our security team and then monitored until they were resolved. Unresolved bugs were eligible for the Patch Rewards Program. While this reporting process had some success, it was overly complex. Now, by unifying and automating our fuzzing tools, we have been able to consolidate our processes into a single workflow, based on OSS-Fuzz. Projects integrated with OSS-Fuzz will benefit from being reviewed by both our internal and external fuzzing tools, thereby increasing code coverage and discovering bugs faster.
We are committed to helping open source projects benefit from integrating with our OSS-Fuzz fuzzing infrastructure. In the coming weeks, we will reach out via email to critical projects that we believe would be a good fit and support the community at large. Projects that integrate are eligible for rewards ranging from $1,000 (initial integration) up to $20,000 (ideal integration); more details are available here. These rewards are intended to help offset the cost and effort required to properly configure fuzzing for OSS projects. If you would like to integrate your project with OSS-Fuzz, please submit your project for review. Our goal is to admit as many OSS projects as possible and ensure that they are continuously fuzzed. Once contacted, we might provide a sample fuzz target to you for easy integration. Many of these fuzz targets are generated with new technology that understands how library APIs are used appropriately. Watch this space for more details on how Google plans to further automate fuzz target creation, so that even more open source projects can benefit from continuous fuzzing.
Thank you for your continued contributions to the Open Source community. Let’s work together on a more secure and stable future for Open Source Software.
Today, we’re excited to introduce reCAPTCHA v3, our newest API that helps you detect abusive traffic on your website without user interaction. Instead of showing a CAPTCHA challenge, reCAPTCHA v3 returns a score so you can choose the most appropriate action for your website.
A frictionless user experience Over the last decade, reCAPTCHA has continuously evolved its technology. In reCAPTCHA v1, every user was asked to pass a challenge by reading distorted text and typing into a box. To improve both user experience and security, we introduced reCAPTCHA v2 and began to use many other signals to determine whether a request came from a human or bot. This enabled reCAPTCHA challenges to move from a dominant to a secondary role in detecting abuse, letting about half of users pass with a single click. Now with reCAPTCHA v3, we are fundamentally changing how sites can test for human vs. bot activities by returning a score to tell you how suspicious an interaction is and eliminating the need to interrupt users with challenges at all. reCAPTCHA v3 runs adaptive risk analysis in the background to alert you of suspicious traffic while letting your human users enjoy a frictionless experience on your site.
More Accurate Bot Detection with "Actions"
In reCAPTCHA v3, we are introducing a new concept called “Action”—a tag that you can use to define the key steps of your user journey and enable reCAPTCHA to run its risk analysis in context. Since reCAPTCHA v3 doesn't interrupt users, we recommend adding reCAPTCHA v3 to multiple pages. In this way, the reCAPTCHA adaptive risk analysis engine can identify the pattern of attackers more accurately by looking at the activities across different pages on your website. In the reCAPTCHA admin console, you can get a full overview of reCAPTCHA score distribution and a breakdown for the stats of the top 10 actions on your site, to help you identify which exact pages are being targeted by bots and how suspicious the traffic was on those pages.
Fighting bots your way
Another big benefit that you’ll get from reCAPTCHA v3 is the flexibility to prevent spam and abuse in the way that best fits your website. Previously, the reCAPTCHA system mostly decided when and what CAPTCHAs to serve to users, leaving you with limited influence over your website’s user experience. Now, reCAPTCHA v3 will provide you with a score that tells you how suspicious an interaction is. There are three potential ways you can use the score. First, you can set a threshold that determines when a user is let through or when further verification needs to be done, for example, using two-factor authentication and phone verification. Second, you can combine the score with your own signals that reCAPTCHA can’t access—such as user profiles or transaction histories. Third, you can use the reCAPTCHA score as one of the signals to train your machine learning model to fight abuse. By providing you with these new ways to customize the actions that occur for different types of traffic, this new version lets you protect your site against bots and improve your user experience based on your website’s specific needs.
In short, reCAPTCHA v3 helps to protect your sites without user friction and gives you more power to decide what to do in risky situations. As always, we are working every day to stay ahead of attackers and keep the Internet easy and safe to use (except for bots).
Ready to get started with reCAPTCHA v3? Visit our developer site for more details.
Posted by Per Bjorke, Product Manager, Ad Traffic Quality
Fighting invalid traffic is essential for the long-term sustainability of the digital advertising ecosystem. We have an extensive internal system to filter out invalid traffic – from simple filters to large-scale machine learning models – and we collaborate with advertisers, agencies, publishers, ad tech companies, research institutions, law enforcement and other third party organizations to identify potential threats. We take all reports of questionable activity seriously, and when we find invalid traffic, we act quickly to remove it from our systems.
Last week, BuzzFeed News provided us with information that helped us identify new aspects of an ad fraud operation across apps and websites that were monetizing with numerous ad platforms, including Google. While our internal systems had previously caught and blocked violating websites from our ad network, in the past week we also removed apps involved in the ad fraud scheme so they can no longer monetize with Google. Further, we have blacklisted additional apps and websites that are outside of our ad network, to ensure that advertisers using Display & Video 360 (formerly known as DoubleClick Bid Manager) do not buy any of this traffic. We are continuing to monitor this operation and will continue to take action if we find any additional invalid traffic.
While our analysis of the operation is ongoing, we estimate that the dollar value of impacted Google advertiser spend across the apps and websites involved in the operation is under $10 million. The majority of impacted advertiser spend was from invalid traffic on inventory from non-Google, third-party ad networks.
A technical overview of the ad fraud operation is included below.
Collaboration throughout our industry is critical in helping us to better detect, prevent, and disable these threats across the ecosystem. We want to thank BuzzFeed for sharing information that allowed us to take further action. This effort highlights the importance of collaborating with others to counter bad actors. Ad fraud is an industry-wide issue that no company can tackle alone. We remain committed to fighting invalid traffic and ad fraud threats such as this one, both to protect our advertisers, publishers, and users, as well as to protect the integrity of the broader digital advertising ecosystem. Technical Detail Google deploys comprehensive, state-of-the-art systems and procedures to combat ad fraud. We have made and continue to make considerable investments to protect our ad systems against invalid traffic.
As detailed above, we’ve identified, analyzed and blocked invalid traffic associated with this operation, both by removing apps and blacklisting websites. Our engineering and operations teams, across various organizations, are also taking systemic action to disrupt this threat, including the takedown of command and control infrastructure that powers the associated botnet. In addition, we have shared relevant technical information with trusted partners across the ecosystem, so that they can also harden their defenses and minimize the impact of this threat throughout the industry.
The BuzzFeed News report covers several fraud tactics (both web and mobile app) that are allegedly utilized by the same group. The web-based traffic is generated by a botnet that Google and others have been tracking, known as “TechSnab.” The TechSnab botnet is a small to medium-sized botnet that has existed for a few years. The number of active infections associated with TechSnab was reduced significantly after the Google Chrome Cleanup tool began prompting users to uninstall the malware.
In similar fashion to other botnets, this operates by creating hidden browser windows that visit web pages to inflate ad revenue. The malware contains common IP based cloaking, data obfuscation, and anti-analysis defenses. This botnet drove traffic to a ring of websites created specifically for this operation, and monetized with Google and many third party ad exchanges. As mentioned above, we began taking action on these websites earlier this year.
Based on analysis of historical ads.txt crawl data, inventory from these websites was widely available throughout the advertising ecosystem, and as many as 150 exchanges, supply-side platforms (SSPs) or networks may have sold this inventory. The botnet operators had hundreds of accounts across 88 different exchanges (based on accounts listed with “DIRECT” status in their ads.txt files).
This fraud primarily impacted mobile apps. We investigated those apps that were monetizing via AdMob and removed those that were engaged in this behavior from our ad network. The traffic from these apps seems to be a blend of organic user traffic and artificially inflated ad traffic, including traffic based on hidden ads. Additionally, we found the presence of several ad networks, indicating that it's likely many were being used for monetization. We are actively tracking this operation, and continually updating and improving our enforcement tactics.