Audit Director – AI Non-model Objects

Discover your future at Citi

Working at Citi is far more than just a job. A career with us means joining a team of more than 230,000 dedicated people from around the globe. At Citi, you’ll have the opportunity to grow your career, give back to your community and make a real impact.

Job Overview

Whether you’re at the start of your career or looking to discover your next adventure, your story begins here.  At Citi, you’ll have the opportunity to expand your skills and make a difference at one of the world’s most global banks.  We’re fully committed to supporting your growth and development from the start with extensive on-the-job training and exposure to senior leaders, as well as more traditional learning.  You’ll also have the chance to give back and make a positive impact where we live and work through volunteerism.

The Audit Director – AI Non-model Objects is a senior level management position, reporting to the Chief Auditor Artificial Intelligence (AI) Assurance, responsible for managing a team of audit professionals with an overall objective to strategically direct audit activities and assessments of the risk and control environment for the adoption of AI/GenAI, specifically related to training and use of GenAI models and their related components such as LLMs, tokenizers, embedding , inference and serving infrastructure (Non-model Objects).

This role will primarily focus on AI Non-model Objects with the overall objective of this role to ensure that Citi Internal Audit’s approach to AI to position Citi IA as the leading IA function in the financial services industry.

What you will do

• Develop the strategy and direct the execution of a portfolio of concurrent assurance activities relating to non AI model objects development, validation and deployment, of high quality, within budget, and within the expected timelines.

• Act as a trusted subject matter experts on non AI model risks and controls in financial services across the IA team.

• Partner with IA Model Risk team to scale AI assurance across IA work programs.

• Evaluate AI use cases (design, prompts, embeddings, performance) against technical standards, guidelines, and pipeline integrity.

• Assessing AI/GenAI use case lifecycle management across development, validation, deployment, monitoring, and retirement.

• Evaluate risk dimensions including bias, fairness, robustness, explainability, and transparency for non-AI model objects

• Assess guardrail design and effectiveness to mitigate hallucinations, prompt injection, and unsafe outputs.

• Provide technical assurance input into model-related regulatory reviews and compliance assessments.

• Ensure timely delivery of comprehensive regulatory and internal audit issue validation, including issues arising from other external parties.

• Evaluate the adequacy and effectiveness of internal AI governance policies, adherence to AI/GenAI related laws and regulations, and the robustness of AI-related ethical or compliance incident response frameworks.

• Evaluate testing methods and approaches, assess adequacy and completeness of tests given the specific needs for GenAI systems and other non model AI objects.

• Evaluate functional red-teaming approaches for AI use-cases and non-model objects  

• Attract, develop, and retain talent; recommending staffing levels required to fulfil responsibilities effectively while establishing and adhering to talent management processes and compensation and performance management programs.

• Work across IA to maximize the efficiency and effectiveness of resources.

• Provide local site leadership encouraging engagement and collaboration across all IA organizations.

• Perform control environment assessments and identify thematic issues and trends to improve controls and governance over the identification, measurement, management, and reporting of risks.

• Champion horizon scanning for emerging methods and approaches for GenAI system risk management

• Promote auditability by design by advising product and engineering teams on controls, documentation, and governance structure needed to make AI systems inherently auditable.

• Shape the future of AI audit practices by contributing to the development of new assurance methods for emerging risks (systemic bias propagation, synthetic data misuse, model drift)

• Support and endorse the Quality Assurance (QA) function of IA, and resolve issues found by QA, improving audit processes and coverage.

• Represent IA in relevant governance forums including Boards and Committees.

• Develop and maintain strong working relationships across other IA teams, Management Stakeholders (MSs), and Regulators.

• Build trust through proactive and transparent interactions while maintaining strong relationships with regulators aligned to Internal Audit.

• Engage regularly with MSs to understand emerging risks and help address escalations.

• Pro-actively advise and assist MSs in considering risks and controls. Support the audit team in their interactions with MSs.

• Participate and share relevant knowledge and insight, on an ongoing basis, within and across IA teams, to drive the consistent application of the methodology, consistency of audit opinions, higher efficiency, and improved quality of overall assurance.

What we will need from you

Qualifications

• 15+ years experience in model audit, validation, or model risk management within financial services.

• Experience of LLMs, GenAI validation and testing methods

• Understanding of financial regulations and how they intersect with AI/GenAI e.g. conduct risk, operational resilience, data protection, algorithmic trading, AML/KYC, consumer protection.

• Preferably with audit experience, subject matter expert in model validation and governance.

• Related certifications such as Certified Information Systems (CISA), Certified Internal Auditor (CIA), Certified in Risk and Information Systems (CRISC) or similar.  AI-related certifications are a plus.

• Executive presence and expert in building and maintaining strong open relationships with executive stakeholders, working as a partner, exerting influence and providing credible challenge in a constructive manner.

• Ability to quickly comprehend the full risk implications of complex global issues, escalate to the appropriate level and provide advice and on pragmatic commercial solutions.

• Outstanding performer, open minded, resilient, agile, energetic, self-starter, articulate and empathetic whilst being confident to deliver opinions to bring about positive outcomes.

• Strong leadership skills including a track record of identifying and developing world class talent. 

• Experienced in leading large, dispersed and diverse professional teams.

• Demonstrated success in business, functional and people management.

• Consistently demonstrates clear and concise written and verbal communication.

• Demonstrated ability to remain unbiased in a diverse working environment.

• Effective negotiation skills.

Education

• Bachelor’s degree/University degree in decision science, computer science, data science, mathematics, statistics or a related field, or equivalent experience.

• Master’s degree preferred.

This job description provides a high-level review of the types of work performed. Other job-related duties may be assigned as required.

Working at Citi is far more than just a job. A career with us means joining a family of more than 230,000 dedicated people from around the globe. At Citi, you’ll have the opportunity to grow your career, give back to your community and make a real impact.

Take the next step in your career, apply for this role at Citi today

https://jobs.citi.com/dei

------------------------------------------------------

Job Family Group:

Internal Audit

------------------------------------------------------

Job Family:

Audit

------------------------------------------------------

Time Type:

Full time

------------------------------------------------------

Most Relevant Skills

Control Monitoring, Data Analysis, Industry Knowledge, Issue Management, Laws and Regulations, Professional Standards, Risk Management, Root Cause Analysis.

------------------------------------------------------

Other Relevant Skills

Generative AI, Internal Audit Development, Internal Audit Management, Model Audit, Model Risk, Model Validation.

------------------------------------------------------

Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.

If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review Accessibility at Citi.

View Citi’s EEO Policy Statement and the Know Your Rights poster.