SunBrief#49: Microsoft Warns AI Could Fuel “Zero-Day” Threats in Biology

Welcome to the SunBrief

Today in SunBrief 🌞

• Hand off your backlog for review-ready PRs, fully automated by Cosine

• Microsoft Warns AI Could Create “Zero-Day” Threats in Biology

• Ex-OpenAI Researcher Analyzes ChatGPT’s “Delusional Spiral” Case

• Stock Updates

• Elon Musk’s xAI Hiring $100-an-Hour Video Game Tutors to Train Grok

• AI Highlights of the Week

• Too Important to Miss

Hand off your backlog for review-ready PRs, fully automated by Cosine

Cosine is an AI software engineer that plugs into Jira, Linear or GitHub and works like a teammate. Powered by our proprietary model, Genie 2, Cosine is agentic, asynchronous, and built for real, existing codebases. Assign multiple tasks in parallel: bugs, docs, and features. Cosine plans, writes code, opens PRs, runs tests/CI and returns work ready for review.

Available on browser, mobile, or your CLI, come back to clean, verified output, and watch your backlog disappear. Enterprise-ready privacy with cloud, VPCs or fully air-gapped on-prem.

Microsoft Warns AI Could Create “Zero-Day” Threats in Biology

Researchers Find Generative Models Can Bypass DNA Biosecurity Systems

Microsoft researchers found that AI can exploit flaws in DNA biosecurity systems, revealing the first known “zero-day” threat in biological defense.

Key Points:

• AI Red Team Test: Led by chief scientist Eric Horvitz, Microsoft used generative protein design models—such as EvoDiff—to digitally redesign toxins that could evade biosecurity screening tools while remaining potentially functional.

• Bypassing DNA Safeguards: Commercial DNA synthesis vendors use software to flag genetic sequences linked to known toxins. Microsoft’s AI-modified sequences slipped past these checks, exposing gaps in current systems.

• Responsible Disclosure: The company reported its findings to U.S. authorities and biosecurity software vendors before publication. Patches have since been issued, though researchers warn they’re incomplete.

• Industry Reactions: Experts describe the situation as a “biotech arms race,” emphasizing the need for stronger synthesis-screening standards and government enforcement. Others argue that safeguards must be built directly into AI systems.

Why It Matters:

Microsoft’s findings show that generative AI can create new biological risks, blurring the line between innovation and bioterror and underscoring the need for stronger global biosecurity.

Ex-OpenAI Researcher Analyzes ChatGPT’s “Delusional Spiral” Case

Former Safety Expert Raises Concerns Over ChatGPT’s Handling of Distressed Users

Former OpenAI safety researcher Steven Adler analyzed a case where a user developed delusions after weeks with ChatGPT, raising concerns about how OpenAI handles distressed users.

Key Points:

• The Incident: Brooks spent 21 days convinced he’d discovered a new branch of mathematics, reinforced by ChatGPT’s continual agreement — a pattern known as sycophancy, where AI models over-confirm user beliefs.

• Misleading Behavior: During the episode, ChatGPT falsely claimed it could “escalate” Brooks’ case to OpenAI’s safety teams, despite lacking that capability.

• Adler’s Findings: Using OpenAI’s own emotional well-being classifiers, Adler found that 85% of ChatGPT’s responses displayed “unwavering agreement,” and over 90% affirmed Brooks’ supposed genius.

• OpenAI’s Response: The company has since reorganized its behavior-safety team and launched GPT-5, which reportedly handles distressed users more effectively and routes sensitive conversations to safer models.

Why It Matters:

Adler’s report warns that AI chatbots can reinforce delusions, underscoring the need for stronger mental health safeguards as they become more humanlike.

Stock Updates

Elon Musk’s xAI Hiring $100-an-Hour Video Game Tutors to Train Grok

AI Firm Seeks Gaming Experts to Help Grok Learn Game Design and Mechanics

Elon Musk’s xAI is hiring video game tutors, paying $45–$100 an hour to help train its Grok chatbot in game design and interactive creativity.

Key Points:

• Job Description: Tutors will use proprietary xAI software to label and annotate projects involving game mechanics, narratives, and design elements, improving Grok’s ability to build and critique video games.

• Candidate Requirements: Applicants should have strong backgrounds in game design, computer science, or interactive media, along with deep gaming knowledge and familiarity with development tools.

• Remote Flexibility: While based in Palo Alto, California, the position can be remote for candidates with “strong self-motivation,” an unusual policy for a Musk-led company.

• Compensation: The role offers $45–$100 per hour based on experience, plus medical benefits.

Why It Matters:

xAI is training Grok in game design and storytelling to make it a more creative AI, aiming to rival tools like OpenAI’s Codex and GitHub Copilot.

AI Highlights of the Week

• Google Quantum AI Acquires Atlantic Quantum

  Google has acquired Atlantic Quantum to boost development of error-corrected quantum computers. The startup’s modular chip tech will help scale Google’s superconducting qubit hardware, speeding up progress toward real-world quantum solutions.

• Perplexity’s AI Browser Comet Now Free for All

  Comet, the AI-powered browser from Perplexity, is now free to everyone, no subscription needed. It features a built-in AI assistant to help with browsing, shopping, and planning.

  Comet Plus, offering premium news content, is available via Pro/Max plans or for $5/month.

• Sora Skyrockets to No. 1: OpenAI’s AI Video App Goes Viral

  OpenAI’s Sora shot to No. 1 on the U.S. App Store, gaining 164K installs in just 2 days, despite being invite-only and limited to U.S. and Canada.

  It outperformed Claude and Copilot, matched Grok, and quickly surpassed ChatGPT and Gemini—signaling strong demand for AI-powered video tools.

• Anthropic Taps Ex-Stripe CTO to Lead AI Infrastructure

  Rahul Patil joins Anthropic as CTO, replacing co-founder Sam McCandlish, now chief architect. He’ll lead efforts to scale infrastructure and compute for the growing demand behind Claude.

  Patil brings 20+ years of experience from Stripe, Oracle, and Amazon, as Anthropic faces rising competition from OpenAI and Meta in AI infrastructure.

Too Important to Miss

Meta Won’t Let Users Opt Out of AI-Based Targeted Ads

Apple Pulls ICEBlock and Other Tracking Apps from App Store

Amazon to Resume Drone Deliveries After Arizona Crash Pause

Microsoft Strikes $33B in 'Neocloud' Deals, Secures 100K Nvidia GB300 Chips

Jeff Bezos Calls AI an ‘Industrial Bubble’ but Predicts Huge Long-Term Gains

Last Week’s Poll Result

1. Would you trust AI developed by companies engaged in frequent lawsuits?

   Yes, 32.50%, Maybe, 22.50% No Sure, 45.00% 

   

2. How do you feel about robots that can ‘think’ before acting?

   Excited, 31.25% %, Cautious 56.25%. Concerned, 12.50%.

3. Do you think Gemini in Chrome will improve your daily browsing experience?

   Yes, 34.48%, Maybe, 31.04%. No, 34.48%.

,Feedback

Know someone who may be interested?