Senior Linux Administrator

GovCIO is seeking a highly experienced Senior Linux Administrator to serve as the primary technical resource responsible for theadministration, sustainment, security, automation, and optimization of enterprise Red Hat Enterprise Linux (RHEL) systems supporting the616th Operations Center (616 OC). This position is located in San Antonio, TX and is onsite.

This is a senior, hands-on position operating in a VMware-based enterprise environment. The successful candidate must be capable ofindependently owning the Linux environment, resolving complex technical issues, implementing secure and repeatable administrativeprocesses, and serving as the primary escalation point for Linux-related systems and services.

The position requires a self-starter who can operate with limited technical oversight, identify problems before they become operationalimpacts, develop solutions, and drive those solutions through implementation, validation, automation, and documentation.

• Responsibilities
  • Serve as the primary technical owner and senior administrator for enterprise Red Hat Enterprise Linux systems.
  • Install, configure, maintain, patch, upgrade, troubleshoot, secure, and optimize RHEL servers and supporting infrastructure.
  • Administer Red Hat Satellite infrastructure supporting system registration, content management, repositories, lifecycleenvironments, patching, provisioning, and configuration management.
  • Plan and execute Linux operating system lifecycle activities, including provisioning, upgrades, migrations, patching, vulnerabilityremediation, and decommissioning.
  • Apply and maintain Defense Information Systems Agency (DISA) Security Technical Implementation Guide (STIG) configurationsacross Linux systems.
  • Analyze STIG findings, vulnerability scan results, and cybersecurity findings and independently develop appropriate technicalremediation.
  • Develop and maintain Bash scripts used to automate administrative, security, patching, monitoring, compliance, and maintenanceactivities.
    Implement and support configuration management and infrastructure automation capabilities.
  • Work closely with VMware administrators to deploy, configure, troubleshoot, and optimize Linux virtual machines hosted withinVMware vSphere/vCenter environments.
  • Support Linux-hosted enterprise applications and services, including Elastic Stack components, and coordinate operating systemand application-layer troubleshooting when issues cross technical boundaries.
  • Troubleshoot complex operating system, application, storage, networking, authentication, performance, and security issues.
  • Perform root-cause analysis for recurring or high-impact incidents and implement corrective actions that reduce repeat failures.
  • Manage system services, processes, logs, scheduled jobs, storage, networking, repositories, certificates, security controls, and useraccess.
  • Maintain enterprise Linux repositories and package management configurations.
  • Develop and maintain standardized server configurations, build procedures, automation, technical documentation, StandardOperating Procedures (SOPs), and troubleshooting guides.
  • Identify opportunities to eliminate repetitive manual administration through scripting, configuration management, andautomation.
  • Monitor system availability, utilization, capacity, and performance and proactively address identified deficiencies.
  • Support backup, restoration, continuity, and disaster-recovery requirements for Linux systems.
  • Coordinate Linux changes and maintenance with cybersecurity, virtualization, networking, application, engineering, and mission-system teams.
  • Evaluate new Red Hat technologies and capabilities and recommend improvements to the existing Linux environment.
  • Provide technical leadership for Linux-related projects, upgrades, modernization efforts, and engineering activities.

High School with 9+ years (or commensurate experience)

Required Skills and Experience
Clearance Required: TS/SCI with NATO Secret

Candidates must demonstrate significant hands-on experience performing enterprise Linux administration. General IT support experience,application administration, or limited exposure to Linux is not sufficient for this position.

• DoD 8140 – 451 System Administrator IT (Cyberspace) certification(s) (Cloud+ - CompTIA Cloud plus; GICSP - Global Industrial Cyber Security Professional; GSEC - GIAC Security Essentials Certification; Security+ - CompTIA Security plus; and/or SSCP - Systems Security Certified Practitioner) and/or affiliated training as identified within the DoD 8140 Qualification Matrix (https://www.cyberworkforce.mil/DoD-8140/8140-Dashboard/)“IT (Cyberspace) – 451 System Administrator; Intermediate
• 7+ years of hands-on enterprise Linux systems administration experience, including substantial recent experience administeringRed Hat Enterprise Linux (RHEL) in production environments.
• Demonstrated experience serving as a senior administrator, technical lead, primary escalation resource, or technical owner forproduction Linux systems.
• Ability to independently administer and troubleshoot Linux systems from the command line without reliance on a graphical userinterface.
• Demonstrated hands-on administration of Red Hat Satellite, including host registration, repositories, Content Views, LifecycleEnvironments, Activation Keys, subscription/manifests, synchronization, package/patch deployment, and troubleshooting contentor registration failures.
• Working knowledge of Satellite command-line administration using Hammer CLI or equivalent methods. Experience with SatelliteCapsules and distributed content architectures is advantageous.
• Demonstrated experience performing enterprise RHEL patch management, including planning, testing, deployment, validation,remediation of failed updates, and rollback/recovery considerations.
• Demonstrated hands-on implementation and remediation of DISA STIG requirements on RHEL systems. Familiarity with STIGterminology alone is not sufficient.
• Experience reviewing vulnerability or compliance findings and implementing operating-system-level remediation in coordinationwith cybersecurity personnel.
• Strong knowledge of Linux security controls and administration, including SELinux, firewalld, SSH, sudo, PAM, permissions, auditing,account management, service hardening, and certificate management.
• Strong Bash scripting proficiency with demonstrated ability to create and maintain scripts used for systems administration andautomation.
• Ability to write Bash scripts using variables, loops, conditionals, functions, error handling, exit codes, logging, text processing, andcommand-output validation.
• Demonstrated experience using configuration management, orchestration, or infrastructure automation technologies such asAnsible, SaltStack, Puppet, Chef, or equivalent.
• Strong understanding of RHEL package and repository management, including RPM, DNF/YUM, .repo configuration, packagedependencies, and local or enterprise repositories.
• Demonstrated experience administering Linux virtual machines in a VMware vSphere/vCenter environment, including provisioning,compute/memory allocation, virtual networking, storage interaction, snapshots, and VMware/Open VM Tools.
• Strong Linux troubleshooting skills across operating system, compute, memory, disk, I/O, application, network, authentication, andservice layers.
• Strong understanding of core Linux administration, including systemd, journald/system logging, cron/systemd timers, kernel andOS parameters, filesystems, process management, and boot/recovery procedures.
• Demonstrated experience managing Linux storage, including partitions, Logical Volume Manager (LVM), filesystemcreation/expansion, mounts, NFS, disk utilization, and storage troubleshooting.
• Strong understanding of enterprise networking concepts as they relate to Linux administration, including TCP/IP, DNS, routing,subnets, VLANs, NTP, firewalls, ports, and protocols.
• Experience integrating Linux systems with enterprise authentication and directory services such as Microsoft Active Directory, LDAP,Kerberos, or SSSD.
• Experience managing certificates, certificate chains, trust stores, and TLS-enabled Linux services.
• Experience monitoring Linux system health and analyzing CPU, memory, disk, I/O, process, network, and application performance.
• Experience performing system backup, restoration, recovery, and troubleshooting activities.
• Demonstrated ability to independently investigate complex technical issues, identify root causes, develop corrective actions,implement solutions, validate results, and document the outcome.
• Demonstrated ability to manage multiple technical priorities while maintaining production system availability and security.
• Strong technical documentation skills, including development and maintenance of procedures, system configurations,troubleshooting documentation, and SOPs.
• Strong written and verbal communication skills and the ability to coordinate effectively with cybersecurity, virtualization,networking, application, engineering, and customer teams.

Preferred Skills and Experience
The following capabilities distinguish especially strong candidates but are not substitutes for the required Linux administration experienceabove.

• Advanced Ansible experience, including development and maintenance of playbooks, roles, inventories, variables, templates,handlers, reusable automation, and automated configuration/compliance workflows.
• Experience integrating Ansible with Red Hat Satellite or other enterprise Linux management platforms.
• Python proficiency, particularly development of Python utilities used for systems administration, API integration, data processing,reporting, or automation.
• Experience consuming or integrating REST APIs as part of infrastructure automation.
• Experience with Git-based source control and maintaining infrastructure scripts and configuration artifacts through controlledversion management.
• Experience implementing automated STIG or security-compliance validation and remediation.
• Experience with security tools such as OpenSCAP, SCAP Compliance Checker, STIG Viewer, ACAS/Nessus, or comparable securityassessment technologies.
• Experience with automated RHEL provisioning using Kickstart, PXE, DHCP, TFTP, Satellite provisioning, or comparable technologies.
• Experience designing and maintaining standardized RHEL server builds or golden images.
• Demonstrated experience creating, maintaining, validating, publishing, and deploying Linux VDI golden images and instant-clonedesktop pools in an Omnissa Horizon environment.
• Experience supporting the Linux VDI image lifecycle, including Horizon agent integration, image updates, pool deployment, imagepush/publishing operations, Active Directory/SSSD integration, application packaging, profile considerations, and troubleshootingclone or registration failures.
• Experience troubleshooting Linux virtual desktops across RHEL, Omnissa Horizon, vSphere/vCenter, virtual networking,authentication, and application layers.
• Production experience administering or supporting Elastic Stack services hosted on Linux, including Elasticsearch, Kibana, Logstash,Beats, and/or Elastic Agent.
• Experience diagnosing Elastic cluster health and performance issues, including node health, shard allocation, replicas, JVM/heappressure, disk watermarks, indexing/search performance, and service/log analysis.
• Experience with Elastic security and lifecycle operations, including TLS certificates, role-based access controls, snapshots/recovery,upgrades, index lifecycle management (ILM), and configuration changes.
• Experience automating Elastic deployment, configuration, patching, or health validation using Ansible, Bash, Python, or comparabletools.
• Experience with Elastic ingestion and management capabilities such as Logstash pipelines, ingest pipelines, Fleet, integrations,index templates, or similar capabilities.
• Experience with enterprise monitoring platforms such as Elastic, Nagios, Zabbix, Splunk, or equivalent.
• Experience performing RHEL major-version migrations or large-scale operating system lifecycle upgrades.
• Experience supporting disconnected, restricted, classified, or air-gapped Linux environments.
• Experience supporting Department of Defense or Intelligence Community production environments.
• Familiarity with Risk Management Framework (RMF), Authority to Operate (ATO), vulnerability remediation, Plans of Action andMilestones (POA&M), and cybersecurity change-control processes.
• Experience designing automation to reduce administrative toil, improve configuration consistency, and increase system reliability.
• Red Hat Certified System Administrator (RHCSA) certification.
• Red Hat Certified Engineer (RHCE) certification.
• Red Hat Certified Specialist certifications related to Satellite, Ansible Automation, security, or enterprise deployment.
• VMware Certified Professional (VCP-DCV or equivalent) certification.