Software Engineer - AI Infrastructure

We’re seeking a software engineer to support our AI infrastructure team at Neural Solutions, where you’ll help build and maintain the foundation for advanced AI capabilities while ensuring access to high-quality LLMs across the inference software stack.

Responsibilities:

Procure, configure, and test new inference models for release to users.
Develop in-house services and techniques to ensure continuous high-quality inference service.
Work with model vendor teams to create reliable pipelines for closed-source model usage.
Collaborate on surge efforts to support short-term, high-priority inference needs.
Engage with other teams to establish robust infrastructure and integrate LLM-powered tools.

Required Skills:

Experience with Python or other modern programming languages.
Familiarity with Argo CD or CI/CD frameworks.
Experience with Kubernetes/Helm.
Familiarity with AWS or other cloud service providers.
Able to quickly learn new technologies and have strong communication skills.

Nice to Have:

Experience with vLLM, LiteLLM, or similar inference-serving frameworks.
Familiarity with LLM hosting practices.
Knowledge of SRE best practices for production software support.
Experience with Elastic, Grafana/Prometheus, or other observability practices.
Background in Docker and containerization.
Understanding of traffic shaping and quality-of-service engineering.
Knowledge of hosting AI capabilities.

Experience Required: 3 years with Bachelor’s degree in technical field, or 7 years without degree

Location: Columbia, MD

Clearance: TS/SCI with Polygraph required

Salary Range: $138,000 - $163,000

Junior Software Engineer – Inference