Adult, Female, Person, Woman, Cup, Glasses, Boy, Male, Teen, Bracelet

Principal Engineer, LLM

Principal Engineer, LLM

  • R-00507
  • Remote - United States, United States
  • Machine Learning Department

About Upstart

At Upstart, we’re united by a mission that matters: to radically reduce the cost and complexity of borrowing for all Americans. Every day, we bring creativity, experimentation, and advanced AI to reshape access to credit, helping millions move forward financially with clarity and confidence.

As the leading AI lending marketplace, we partner with banks and credit unions to expand access to affordable credit through technology that’s both radically intelligent and deeply human. Our platform runs over one million predictions per borrower using more than 3,000 signals, powering smarter, fairer decisions for millions of customers. But the numbers only hint at the impact. Every idea, every voice, and every contribution moves us closer to a world where credit never stands between people and their financial progress.

We’re proudly digital-first, giving most Upstarters the flexibility to do their best work from wherever they thrive, alongside teammates across 80+ cities in the US and Canada. Digital-first doesn’t mean distant. We’re intentional about in-person connection through team onsites, planning sessions, and moments that spark creativity and trust. And whether you choose to work primarily from home or collaborate in-person from one of our offices in Columbus, Austin, the Bay Area, or New York City, you’ll have the support to work in the way that works best for you.

If you’re energized by tackling meaningful problems, excited to innovate with purpose, and motivated by work that truly matters, we’d love to hear from you.

 

The Team: 

At Upstart, our Applied AI team's mission is threefold:

  1. Agent Platform: Build and support the platform that makes it straightforward for non technical users across Upstart to build, launch, and operate AI agents (voice/chat/text) reliably.

  2. Borrower and Applicant Assistant Orchestration: Build the orchestration layer behind a unified personal assistant that shares context across specialized agents, giving applicants and borrowers a cohesive, assistant-like experience.

  3. Borrower-Connected Internal Workflows: Improve internal operational workflows that directly connect to applicant and borrower experiences, making them faster, smarter, and more efficient.

As Principal Engineer, LLM you will serve as a technical pathfinder and force multiplier for this team and the product squads we partner with. Rather than owning internal developer-tooling infrastructure or training foundation models from scratch, your mandate is to compress the time between an ambiguous idea and a reliable, user-facing AI system, while raising the applied AI bar across every squad you touch.

This role operates on an explicit 80/20 charter:

  • 0-to-1 Product and Architecture Pathfinder (~80%): Rapidly prototype high-ambiguity bets across our agent platform, cross-agent context orchestration, borrower-facing assistant experiences, and internal workflows. Partner directly with Product and Design to validate non-deterministic UX, establish clean architectural patterns, and hand off battle-tested evaluation contracts for engineering squads to scale.

  • AI-Native Engineering Catalyst (~20%): While dedicated internal teams own Upstart's broader AI developer tooling and software-factory initiatives, you will serve as a visible practitioner on the product side. You will lead by example with agentic coding workflows and tight verification harnesses, partner with our internal tooling teams as an early adopter ("customer zero"), and help product squads adopt higher-leverage ways of working.

Because AI systems are non-deterministic, user experience is part of the architecture. Whether you are designing cross-agent handoffs, borrower-facing assistant flows, or internal operational tools, you will navigate real-world trade-offs across latency, unit economics, reliability, and user trust, and push back constructively when a simpler deterministic approach is the right call.

How you’ll make an impact

  • De-risk 0-to-1 bets across the stack: Rapidly prototype ambiguous initiatives spanning multi-agent context sharing, UX for non technical users, platform primitives for launching agents, and borrower-connected internal workflows.

  • Deliver durable evaluation contracts, not throwaway handoffs: Treat prototype code as disposable proof of concept, and hand off to engineers what they actually need to build production services with confidence: golden datasets, automated evaluation harnesses, documented failure-mode maps, latency/cost benchmarks, and clean architectural blueprints.

  • Partner cleanly from spike to production: Align early with receiving squad tech leads to surface system constraints, support joint go/no-go decisions with Engineering and Product leadership (including killing unviable bets early), and stay engaged through initial productionization to review early PRs and debug edge-case model behavior.

  • Design non-deterministic UX and orchestration together: Treat latency spikes, context loss between agents, subtle hallucinations, and fallback paths as systems design problems, shaping experiences that earn and protect borrower and operator trust.

  • Make pragmatic applied AI trade-offs: Default to the simplest reliable solution first (prompting, context engineering, deterministic guardrails), and make clear architectural calls on model routing, fine-tuning, or distillation when latency, unit economics, or reliability require it.

  • Raise the applied AI bar across squads: Model disciplined agentic coding and verification practices in your daily workflow, feed practical requirements back to internal tooling teams, and elevate engineering judgment across teams through active collaboration.

Minimum Qualifications 

  • Software engineering foundation and live AI shipping: 8+ years of software engineering experience, along with hands-on experience shipping non-deterministic, user-facing LLM or AI products to real users.

  • Agentic and LLM systems architecture: Deep practical experience designing production LLM systems involving structured outputs, tool calling, context engineering, retrieval/memory patterns, and multi-step or multi-agent orchestration (including state management and context sharing across agents).

  • Production systems, latency, and reliability engineering: Strong distributed-systems and backend fundamentals for non-deterministic pipelines: async orchestration, streaming UX, caching, timeouts and fallback paths, rate-limit handling, and end-to-end observability/tracing to diagnose failures.

  • Evaluation and quality engineering: Proven experience building golden datasets, automated evaluation harnesses (deterministic checks, heuristics, and LLM-as-a-judge), regression gates, and systematic failure-mode analyses.

  • Safety, guardrails, and applied AI pragmatism: Strong instinct to reach for the simplest reliable tool first (deterministic logic, prompting, context engineering, guardrails against hallucinations and prompt/data risks), paired with sound judgment on when latency, unit cost, or reliability justify routing, fine-tuning, or distillation over frontier APIs.

  • Full-stack prototyping fluency: Comfortable moving fast across backend and frontend stacks (such as Python, TypeScript/React, and modern REST or streaming APIs) to put working end-to-end prototypes in front of users and stakeholders quickly.

  • Low-ego pathfinder collaboration and UX empathy: Track record aligning early with partner squads, handing off ownership cleanly without ego, and partnering directly with Product, Design, and operations or business leaders to turn ambiguous goals into concrete latency budgets and resilient fallback experiences.

  • Bachelor's degree in Computer Science, Engineering, Mathematics, or equivalent practical experience.

Preferred Qualifications

  • Startup or 0-to-1 incubation DNA: Experience operating in early-stage or low-boundary environments where you owned ambiguous problems end to end under real constraints.

  • Agent platform and shared abstractions: Experience designing reusable agent primitives, SDKs, or orchestration platforms that help peer engineering teams build, evaluate, and launch AI workflows safely and quickly.

  • Production fine-tuning and distillation reps: Hands-on experience fine-tuning or distilling smaller models in production to improve latency, unit economics, or task reliability.

  • AI-native engineering habits: Deep daily fluency with agentic coding workflows and automated verification harnesses, plus experience helping peer engineers adopt higher-leverage development practices.

  • Experience in regulated or high-trust consumer domains (such as fintech, lending, or healthcare) where accuracy, explainability, PII protection, and user trust are critical.

Position location This role is available in the following locations: Remote 

Time zone requirements The team operates on the East/West coast time zones. 

Travel requirements As a digital first company, the majority of your work can be accomplished remotely. The majority of our employees can live and work anywhere in the U.S but are encouraged to to still spend high quality time in-person collaborating via regular onsites. The in-person sessions’ cadence varies depending on the team and role; most teams meet once or twice per quarter for 2-4 consecutive days at a time.

 

#LI-REMOTE

#LI-MidSenior 

 

At Upstart, your base pay is one part of your total compensation package.  The anticipated base salary for this position is expected to be within the below range. Your actual base pay will depend on your geographic location–with our “digital first” philosophy, Upstart uses compensation regions that vary depending on location. Individual pay is also determined by job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.

In addition, Upstart provides employees with target bonuses, equity compensation, and generous benefits packages (including medical, dental, vision, and 401k).

United States | Remote - Anticipated Base Salary Range
$238,400—$330,200 USD

What you'll love

At Upstart, our benefits for full-time employees are designed to support your health, financial well-being, family, and personal growth. Here’s what you can expect:

  • Competitive compensation, including base pay, bonus opportunities, and annual equity grants that vest quarterly 
  • Retirement benefits to help you plan for the future, including a 401(k) or Group Retirement Savings Plan with a company match of $2 for every $1 contributed, up to $15,000 annually (USD in the US, CAD in Canada)
  • Employee Stock Purchase Plan (ESPP) with discounted stock purchase options for eligible employees (US only)
  • Comprehensive health coverage designed to support you and your family, including medical, dental, vision, and wellness resources for US and supplemental health coverage for Canada.
  • Health Savings Account contributions from Upstart for eligible plans (US only)
  • Income protection benefits, including life insurance and disability coverage for added financial security
  • Paid time off, sick leave, and company holidays, in line with local requirements
  • Paid family and parental leave to support caregiving and major life moments (duration varies by country)
  • Family-centered benefits to support fertility, parenthood, and caregiving needs
  • Employee Assistance Program (EAP) offering mental health support and life-centered resources
  • Financial wellness resources, including access to financial planning tools and a financial concierge service (US Only)
  • Annual wellness allowance to support your physical and emotional well-being and personal development, based on what matters most to you
  • Annual productivity allowance to invest in relevant tools and resources you need to do your best work, no matter where you work from
  • Connection and community through team events, all-company updates, and employee resource groups (ERGs)
  • Onsite perks, including catered lunches and fully stocked micro-kitchens when working from one of our offices in the Bay Area, Austin, Columbus, and New York City

For roles based in Canada, please note that we are not currently able to hire in Quebec.

Upstart is a proud Equal Opportunity Employer. Just as we are dedicated to improving access to affordable credit for all, we are committed to inclusive and fair hiring practices.

If you require reasonable accommodation in completing an application, interviewing, completing any pre-employment testing, or otherwise participating in the employee selection process, please email candidate_accommodations@upstart.com

https://www.upstart.com/candidate_privacy_policy

Similar Jobs

There are currently no jobs matching this criteria

Current Upstart employee?

Click Here

Ready to Apply?