Search Jobs

EMBEDDED OS ENGINEER – Linux Firmware

All, Software/SaaS

Salt Lake City, UT

Location: Salt Lake City, UT | Schedule: Full-time, on-site

Overview

An advanced hardware technology company building an autonomous drone platform powered by the Qualcomm QRB5165 processor is seeking a hands-on Embedded OS Engineer. This person will own the Ubuntu-based Linux firmware stack for the platform, ensuring it is secure, reliable, and shipped through automated CI/CD pipelines. The role requires deep comfort with low-level OS internals rather than application-layer development.

What you'll do

  • Own and maintain the build repository, including layer configuration, local.conf tuning, and image recipes used to produce the production firmware image
  • Write and maintain .bb recipes and .bbappend overlays; create .patch files (git diff-based) against the pinned Qualcomm QRB5165 BSP (LU.UM.3.3.1) so upstream sources are never modified directly
  • Debug BitBake task failures across the do_fetch to do_compile to do_rootfs to do_image pipeline, understand sstate-cache behavior, and know when to use -c cleansstate vs. -c rootfs -f
  • Optimize build performance by tuning BB_NUMBER_THREADS and PARALLEL_MAKE, understanding QEMU emulation overhead vs. native ARM64 compilation, and leveraging shared sstate-cache across Jenkins agents
  • Manage package inclusion and exclusion (IMAGE_INSTALL:remove, PACKAGE_EXCLUDE, BAD_RECOMMENDATIONS) and inter-recipe dependency graphs
  • Implement ROOTFS_POSTPROCESS_COMMAND functions for postbuild OS customization, including package upgrades, apt security patching (Ubuntu ESM), service installation, and file system fixups
  • Remove unnecessary services from production images (HTTP servers such as lighttpd, TFTP daemons, and other attack-surface-expanding services) identified via Blue List / Nessus security assessments
  • Write bash scripts and systemd service units that run reliably in a headless, rootfs post-install context with no interactive terminal and limited /proc and /dev availability
  • Manage the dpkg / apt ecosystem within the embedded rootfs, including package pinning, held packages, validating package state, and keeping apt lock files clean across incremental builds
  • Maintain and improve Jenkins pipelines that build Yocto firmware images, including concurrent build isolation, artifact staging, and race-condition handling between parallel jobs
  • Diagnose and resolve Jenkins agent performance issues including Java heap tuning, Docker overlay2 disk pressure, container lifecycle management, and build environment reproducibility
  • Implement proper artifact copy patterns (stageDir isolation) to prevent race conditions when multiple builds run concurrently against shared directories
  • Manage GitLab repository structure and branch protection for the build repo, implementing .patch-based change workflows that preserve the integrity of pinned upstream BSP branches
  • Configure systemd-networkd for runtime network mode switching (DHCP to link-local) on headless devices using udev rules, button-press event handlers, and LED feedback mechanisms
  • Write and maintain systemd service units and timers, understanding WantedBy, After, Requires, and ExecStart semantics for embedded boot sequences
  • Implement reliable USB logging and file transfer services (usb-gadget, udevadm) for field diagnostics on devices with no screen
  • Develop and maintain Prometheus node_exporter integrations and process-exporter configurations for fleet health monitoring via Grafana dashboards
  • Remediate Nessus / Blue List security findings affecting the embedded OS, including SSH hardening (key-only auth, sshd_config tuning), open port reduction, and service inventory documentation
  • Maintain /etc/shadow hygiene, locked account policies, and PAM configurations appropriate for production embedded devices
  • Generate and maintain security compliance artifacts (port inventories, service lists, patch status reports) for internal security assessors
  • Profile and diagnose slow boot sequences, runaway processes, and memory pressure on ARM64 embedded hardware
  • Use screen, tmux, and remote shell tooling to manage long-running build and deployment sessions on headless servers and devices
  • Implement OS-level monitoring including log capture services, boot-time diagnostics, and watchdog patterns for unattended field deployment
  • Advise on cloud build infrastructure choices (AWS Graviton / Azure ARM64) to eliminate QEMU emulation overhead and achieve 3-5x build time reductions

Required qualifications

  • Hands-on experience with Yocto and BitBake build systems, including layer configuration and recipe/overlay development
  • Experience working with Qualcomm QRB5165 or similar embedded BSPs
  • Strong Linux internals knowledge beyond the application layer, including systemd, networking, and package management (dpkg/apt)
  • Experience building and maintaining Jenkins CI/CD pipelines for firmware or embedded builds
  • Experience with GitLab repository management and patch-based workflows
  • Proficiency writing bash scripts and systemd service units for headless embedded environments
  • Experience with OS security hardening, including SSH configuration, PAM, and remediation of Nessus/security scan findings
  • Experience with Docker and container lifecycle management in a build environment
  • Experience with ARM64 architecture and build optimization, including understanding of QEMU emulation overhead
  • Familiarity with Prometheus and Grafana for system monitoring

Why this role

This is a high-ownership position building and securing the core OS platform for an autonomous drone system, combining low-level embedded Linux engineering with full CI/CD pipeline ownership.

"*" indicates required fields

This field is for validation purposes and should be left unchanged.
Address*
Max. file size: 100 MB.
SMS Opt In