Back to Writing
2025-03-01
8 min read

Building Healthcare AI at Scale: Lessons from Y Combinator

Insights from developing AI systems that process 120k+ medical procedures daily and serve 10+ hospitals.

AI/MLHealthcareY Combinator

Building Healthcare AI at Scale: Lessons from Y Combinator

When I joined Mantys Healthcare AI as a Founding Engineer, I knew we were tackling one of the most complex challenges in healthcare: automating prior authorization and claims processing using AI. What I didn't anticipate was the sheer scale and complexity of building systems that now process over 120,000 medical procedures daily across 10+ hospitals.

The Challenge: Healthcare's Data Complexity

Healthcare data is uniquely challenging. Unlike other domains where AI models operate on relatively clean, structured data, healthcare involves:

  • Unstructured medical records with varying formats across providers
  • Complex insurance policies that change frequently and vary by provider
  • Critical accuracy requirements where errors directly impact patient care
  • Regulatory compliance including HIPAA and other healthcare standards

Building Our LLM Evaluation Framework

One of my first major projects was developing a comprehensive evaluation framework for our LLM-based extraction systems. We needed to validate extraction accuracy for copay and eligibility fields with unprecedented precision.

The Technical Stack

We built our evaluation system using:

  • Cove and Gevals for systematic LLM evaluation
  • Log values methods for quantitative accuracy measurement
  • Custom validation pipelines for healthcare-specific edge cases

The results were remarkable: we achieved 98% accuracy while reducing manual verification work by 80%.

Key Learnings

  1. Domain-specific evaluation is crucial: Standard NLP benchmarks don't translate to healthcare
  2. Human-in-the-loop validation: Even with 98% accuracy, human oversight remains essential
  3. Continuous monitoring: Healthcare data evolves rapidly, requiring constant model retraining

Prior Authorization: Replacing Human Expertise with AI

Perhaps our most ambitious project was building a comprehensive Prior Authorization (PA) decision-making system from scratch. This system now automates complex workflows across 120k+ medical procedures.

The Impact

  • Staffing efficiency: Replaced specialized teams of 8-10 PA experts with 2-3 staff members
  • Knowledge democratization: Eliminated reliance on memorized insurance provider policies
  • Consistency: Standardized decision-making across all cases

Technical Architecture

The PA system combines:

  • Multi-modal AI models for processing various document types
  • Rule-based validation for regulatory compliance
  • Real-time API integration with insurance provider systems
  • Audit trails for compliance and quality assurance

Financial Impact: $2.3M in Claims Recovery

Our financial reconciliation system has been one of our most successful implementations, recovering $2.3M in pending claims for healthcare providers.

System Components

  1. Automated status tracking for 50k+ monthly claims
  2. Intelligent reconciliation using pattern recognition
  3. Exception handling for edge cases and errors
  4. Real-time reporting for finance teams

The system reduced reconciliation errors by 90% while dramatically speeding up the revenue cycle.

Y Combinator Experience: Moving Fast in Healthcare

Being part of Y Combinator taught us invaluable lessons about building in the healthcare space:

Speed vs. Compliance

Healthcare moves slowly by necessity, but startups need to move fast. We learned to:

  • Build MVPs quickly while maintaining compliance standards
  • Iterate rapidly within regulatory constraints
  • Focus on user feedback from actual healthcare providers

Scaling Challenges

Growing from a prototype to serving 10+ hospitals taught us:

  • Infrastructure planning is critical for healthcare workloads
  • Data security must be built in from day one
  • Reliability is non-negotiable when patient care is involved

Technologies That Made It Possible

Our tech stack evolved based on healthcare-specific requirements:

# Example of our LLM evaluation pipeline
class HealthcareEvaluationFramework:
    def __init__(self):
        self.cove_evaluator = CoveEvaluator()
        self.gevals_framework = GevalsFramework()
        self.log_analyzer = LogValuesAnalyzer()
        self.domain_validator = HealthcareDomainValidator()

    def evaluate_extraction(self, document, extracted_data):
        # Multi-stage evaluation pipeline
        results = {}

        # Stage 1: Automated evaluation
        results['cove_score'] = self.cove_evaluator.evaluate(document, extracted_data)
        results['gevals_metrics'] = self.gevals_framework.compute_metrics(extracted_data)

        # Stage 2: Domain-specific validation
        results['domain_validation'] = self.domain_validator.validate(extracted_data)

        return results
  • Python + FastAPI: For rapid prototyping and deployment
  • PostgreSQL: For complex healthcare data relationships
  • Celery: For handling long-running medical data processing
  • Playwright: For automated interaction with legacy healthcare systems
  • Docker + AWS: For scalable, compliant infrastructure

Looking Forward: The Future of Healthcare AI

Working at the intersection of AI and healthcare has shown me the immense potential for technology to improve patient outcomes while reducing costs.

Emerging Opportunities

  1. Predictive analytics for patient outcomes
  2. Automated clinical documentation using advanced NLP
  3. Real-time decision support for healthcare providers
  4. Population health management through AI-driven insights

Key Takeaways for Healthcare AI Builders

  1. Accuracy is paramount: Healthcare tolerates no margin for error
  2. Compliance first: Build regulatory requirements into your foundation
  3. Human-centered design: AI should augment, not replace, healthcare professionals
  4. Scalability planning: Healthcare systems need to handle massive, growing datasets

Building healthcare AI at scale has been one of the most challenging and rewarding experiences of my career. The opportunity to directly impact patient care while solving complex technical problems makes every late night debugging session worthwhile.

If you're building in the healthcare AI space, I'd love to connect and share more detailed insights about our technical implementation and lessons learned.

Harshavardhan is a Founding Engineer at Mantys Healthcare AI, a Y Combinator W23 company building AI systems for healthcare automation. Previously, he founded and served as CTO of ReachGig, building full-stack platforms from concept to market.