AI Engineer - Platform

Qualifications

• Strong working knowledge of large language models, retrieval-augmented systems, and common enterprise AI failure modes (hallucinations, overgeneralization, stale context)
• Ability to reason why an AI system produces a given answer and how changes in data, prompts, or configuration affect outcomes
• Experience working with enterprise content systems and a solid understanding of structured vs. unstructured data tradeoffs
• Technical comfort with APIs, integrations, authentication, and data flows, even when working within vendor-managed platforms
• Detail-oriented mindset with a strong bias toward correctness, consistency, and repeatability
• Clear written communication skills, particularly for documenting technical behavior and constraints in an understandable way
• Comfortable operating in an evolving platform environment where the “right” answer may change as models and features evolve
• Sound judgment in balancing platform capability, risk, and operational practicality
• Bachelor’s degree in Computer Science, Information Systems, Data Science, or a related field required
• 3-6 years of experience in software engineering, data engineering, analytics platforms, or AI-adjacent technical roles

Responsibilities

• This role focuses on deeply understanding how large language models, retrieval systems, and enterprise content interact, and ensuring the platform consistently delivers accurate and trustworthy answers aligned with organizational intent
• Reporting to the Director, IT Automation, this individual will own the technical configuration, evaluation, and evolution of the AI platform—ensuring information is indexed appropriately, responses behave as expected, and integrations are designed responsibly
• While not directly responsible for business transformation or change management, this role maintains a strong awareness of why the platform exists and ensures technical decisions support real operational needs
• Own the technical configuration, governance, and ongoing optimization of the enterprise AI platform
• Develop and maintain a deep understanding of LLM behavior, retrieval mechanisms, ranking, and prompt interpretation as they apply to enterprise knowledge and data access
• Evaluate AI-generated responses for accuracy, completeness, grounding, and alignment with expected intent, identifying patterns of failure or degradation
• Establish processes for reviewing prompt behavior and response quality across both knowledge-based and data-oriented queries
• Define and maintain content indexing strategies, including inclusion/exclusion criteria and structural considerations for sources such as SharePoint, Teams, Jira, Confluence, and email
• Design and oversee ingestion and refresh processes to ensure indexed content remains current, relevant, and well-governed
• Support and implement technical integrations that improve platform accessibility and usability, such as Microsoft Teams or other internal tools
• Assess the technical feasibility and implications of integrating additional systems over time (e.g., CRM platforms, financial systems, analytics platforms like Microsoft Fabric)
• Evaluate new platform features, model updates, and vendor capabilities, documenting impact and recommending adoption or mitigation strategies
• Create and maintain technical documentation, platform standards, and operational runbooks related to AI behavior, indexing, and governance
• Support enablement efforts by providing technical guidance and reference materials for users building low/no-code agents within the platform
• While performing the duties of this Job, the employee is occasionally exposed to moving mechanical parts, and fumes or airborne particles

Full Description

The AI Engineer serves as the technical steward of RJW’s enterprise AI platform, responsible for its correctness, reliability, and long-term scalability.

This role focuses on deeply understanding how large language models, retrieval systems, and enterprise content interact, and ensuring the platform consistently delivers accurate and trustworthy answers aligned with organizational intent.

Reporting to the Director, IT Automation, this individual will own the technical configuration, evaluation, and evolution of the AI platform—ensuring information is indexed appropriately, responses behave as expected, and integrations are designed responsibly.

While not directly responsible for business transformation or change management, this role maintains a strong awareness of why the platform exists and ensures technical decisions support real operational needs.

Essential Duties And Responsibilities

• Own the technical configuration, governance, and ongoing optimization of the enterprise AI platform.

• Develop and maintain a deep understanding of LLM behavior, retrieval mechanisms, ranking, and prompt interpretation as they apply to enterprise knowledge and data access.

• Evaluate AI-generated responses for accuracy, completeness, grounding, and alignment with expected intent, identifying patterns of failure or degradation.

• Establish processes for reviewing prompt behavior and response quality across both knowledge-based and data-oriented queries.

• Define and maintain content indexing strategies, including inclusion/exclusion criteria and structural considerations for sources such as SharePoint, Teams, Jira, Confluence, and email.

• Design and oversee ingestion and refresh processes to ensure indexed content remains current, relevant, and well-governed.

• Support and implement technical integrations that improve platform accessibility and usability, such as Microsoft Teams or other internal tools.

• Assess the technical feasibility and implications of integrating additional systems over time (e.g., CRM platforms, financial systems, analytics platforms like Microsoft Fabric).

• Evaluate new platform features, model updates, and vendor capabilities, documenting impact and recommending adoption or mitigation strategies.

• Create and maintain technical documentation, platform standards, and operational runbooks related to AI behavior, indexing, and governance.

• Support enablement efforts by providing technical guidance and reference materials for users building low/no-code agents within the platform.

Knowledge, Skills, And Abilities

• Strong working knowledge of large language models, retrieval-augmented systems, and common enterprise AI failure modes (hallucinations, overgeneralization, stale context).

• Ability to reason why an AI system produces a given answer and how changes in data, prompts, or configuration affect outcomes.

• Experience working with enterprise content systems and a solid understanding of structured vs. unstructured data tradeoffs.

• Technical comfort with APIs, integrations, authentication, and data flows, even when working within vendor-managed platforms.

• Detail-oriented mindset with a strong bias toward correctness, consistency, and repeatability.

• Clear written communication skills, particularly for documenting technical behavior and constraints in an understandable way.

• Comfortable operating in an evolving platform environment where the “right” answer may change as models and features evolve.

• Sound judgment in balancing platform capability, risk, and operational practicality.

Education And Experience

• Bachelor’s degree in Computer Science, Information Systems, Data Science, or a related field required.

• 3-6 years of experience in software engineering, data engineering, analytics platforms, or AI-adjacent technical roles.

• Experience working with enterprise search, knowledge platforms, analytics tools, or AI-powered systems preferred.

• Prior exposure to evaluating or supporting LLM-based solutions in production environments strongly preferred.

• Experience in Warehousing, Logistics, or other operationally complex domains is a plus.

Work Environment

• While performing the duties of this Job, the employee is occasionally exposed to moving mechanical parts, and fumes or airborne particles.

The noise level in the work environment will range from quiet to moderately loud.

Employer Rights

This job description is intended to provide general information about the AI Engineer position.

The above does not constitute an exhaustive list of the job duties to be performed by an associate holding the position of AI Engineer, nor are the lists of the physical requirements and environmental conditions exhaustive.

You may be asked by your supervisor or managers to perform other duties.

Your performance will be evaluated in part based upon your performance of the job duties listed in this job description, as well as any job duties not specifically listed above that you may be asked from time to time to perform.

As with all positions, the duties and responsibilities are subject to change at any time as needs arise and at the discretion of RJW Transport, Inc.

The Company has the right to revise this job description at any time.

Employment-At-Will

It is the Company's policy that all associates, other than those covered by a written individual employment or labor agreement with the Company that has been authorized in writing by the Company's Chief Executive Officer or Board of Directors, are not employed for any fixed term and are employed at the will of the Company for an indefinite period.

Just as our associates reserve the right to resign their employment at any time for any reason the Company reserves its right to terminate an associate at any time for any reason either with or without cause.

Neither this Job Description nor any of its individual terms constitute commitments between the Company and its associates as to the terms, conditions, or duration of employment, nor does it modify the prevailing Employment-At-Will relationship.