Insights

Who knew a single footnote could create so much noise! I'm heading to American Bankers Association Risk and Compliance Conference (RCC) tomorrow. They asked me to speak on genAI and how it is impacting fair lending. I'll be opening up with what I call "The Infamous Footnote". It's a fairly beefy three sentence paragraph at the bottom of a very recent supervision and regulation letter, SR 26-2 put out by the Board of Governors of the Federal Reserve System.

It's funny, I read this whole document with a critical eye, only to get to the end, puzzled. Nothing in the body of the letter said anything at all about generative AI. With my handy friend Claude, I was then led to the page with The Infamous Footnote (highlighting mine).

What is Fair Lending?

Fair lending is the body of law that says credit decisions can't be based on who someone is, only on whether they can repay the loan. It applies to every step of the credit lifecycle. The two main federal statutes are the Equal Credit Opportunity Act (ECOA), which covers all credit, and the Fair Housing Act (FHA), which covers housing-related credit specifically.

To comply with fair lending requirements, you have to design every stage of credit so it doesn't treat protected classes differently and doesn't use "facially neutral" features that act as proxies for protected status. You have to test your actual outcomes for disparities, document why any disparity that exists is justified by business necessity with no less-discriminatory alternative available, and produce specific principal reasons whenever you take adverse action against a consumer. And you have to prove all of that on demand - to examiners, to plaintiffs, to your own board - through inventories, written policies, monitoring records, vendor due diligence, and a paper trail that lets someone reconstruct any single decision from application to outcome.

Looking at a Generative AI Use Case Through a Fair Lending Lens

Let's take a generative example and walk it all the way through. ACME mortgage is releasing a generative pre-qualification chatbot on its website in response to customer requests for more self-service options. The goal of the chat both is to engage conversationally with the human, discern the humans intent, and then provide a set of options for which that human might be eligible. Ultimately, the lender could decide to implement this use case in a lot of different ways - with human in the loop, intersecting LLM with more deterministic solutions, there are many options - for our example, let's say the chatbot is autonomous.

A Generative Example: Pre-Qualification chatbot

The lender would need to design the chatbot so that is does not use things like name or zip code to steer a homeowner into a more expensive product. You need name to pull credit. Zip code is necessary to determine property location and estimate property taxes. Name and zip code are both necessary to pre-qualify a borrower (with credit). Name and zip code are both known "race proxies" in mortgage, variables that looks race-neutral but carry enough racial signal that using it produces the same effect as using race directly. The lender is allowed to use name to pull credit and ZIP code to identify the property and estimate taxes.

What the lender cannot do is let those variables shape the chatbot's product recommendations, rate quotes, or routing — because that's where they would function as race proxies in a way the law treats as discrimination. If the chatbot outputs nonetheless show different outcomes by protected class, the lender's defense is to demonstrate that the design was justified by business necessity and that no less-discriminatory alternative would have served the same need.

In this case, there are two levels of potential for bias to be introduced: (1) the actual application the lender's technology team built to deliver the chatbot to the human, and (2) the underlying foundational model that is discerning intent, potentially deciding request flow, and producing language.

Model and Application Testing

The introduction of generative models like large language models introduces a significant new variable into the testing equation, especially fair lending testing.

When using a large language model at scale, there are two primary roles. The deployer and the developer. The developer is called a foundation model provider - think Open AI with ChatGPT, Anthropic with Claude. They develop the models and make them available for use. YOU as the lender, servicer, regulator, or other mortgage industry actor are the deployer. You are actually giving the model to some set of users to derive benefit for the organization, the homeowners, whomever the target user base is.

As the deployer, you need to understand what types of bias testing the foundation model provider does - the very common ones are BOLD (Bias in Open-ended Language Generation Dataset) and BBQ (Bias Benchmark for QA). It is good to understand these tests, and understand how (or if) they impact your use cases. That's for you to figure out.

Applying the Footnote to Our Use Case

You are solely responsible for fair lending testing on the application, in our example the chatbot. What kind of testing are you obligated to do? This is where the footnote comes in. There is no rulebook for how to test, govern, monitor, or evaluate solutions that incorporate generative solutions. The footnote is really quite unhelpful. First it says generative AI and agentic AI are "out of scope". Then it says your current practices "should guide". And finally it says the principles in the guidance (but not the actual guidance) does apply.

IF, and that's a big if, you decide to implement a pre-qualification chatbot, you should run a standard set of fair lending tests on your chatbot. Follow all the prescribed tests, use all the standard techniques. You should ALSO understand how the generative model is being used in your solution and the extent to which the model capabilities are increasing or decreasing risk. And you should look at what the model providers and any independent benchmarks say about bias testing.

I'll be honest, I don't think an autonomous, generative, pre-qualification chatbot is a good idea right now UNLESS it has a fair amount of determinism in it (so an agentic solution that intersects the intent detection of an LLM with the certainty of a deterministic answering solution).

Closing Thoughts

This is a really challenging topic, with a lot of tentacles. Despite the changes that have happened in the past 12 months, I don't see a responsible way to step back from the traditional practices lenders and servicers have undertaken for fair lending testing. One of the federal requirements has been removed, but the laws are still the laws. The model governance requirements are totally unclear, so we should be cautious in these areas. Be very careful about selecting your use cases. The quadrant chart I provided above gives a good framework, study it carefully and use your best professional (and probably legal) judgment as you make your decisions.

‍

By Tela Mathias, Chief Nerd and Mad Scientist at PhoenixTeam

‍

The thing that keeps me up at night is the future for our kids, and for the young adults trying to get jobs today. Entry-level job postings in the U.S. have diminished by 35% since January 2023. Since the widespread adoption of generative AI, early-career workers (ages 22-25) in the most AI-exposed occupations have experienced a 16 percent relative decline in employment. This AI everything world is like quicksand, shifting as we move. Where will the opportunity be?

Our internship program is kicking off at the tail end of May, and we have the most interns we've ever had - a whopping 16. We dedicated ourselves to creating a program that will prepare these emerging professionals for whatever comes next for them, maybe a job with us, maybe their next year in college, who know. But whatever it is, we want to help them be ready, while also adding value to PhoenixTeam company goals.

We hope to learn from this first group, and roll out a broader set of (free) programs for employers as part of our AI futures initiative. Feel free to take this program framework and use it in your organization. We'd also love to hear from you if you are doing someone innovative in your work or have an idea to share.

The AI Futures Program

A program for preparing people — kids through early-career professionals —to think, work, and lead in a world being rewritten by AI.

The PhoenixTeam Internship Program

The goal of the PhoenixTeam summer internship program is to prepare young adults for whatever their next move towards productive employment is. It is designed for high school, college aged youth, and people with zero to two years of work experience.

The program runs thirteen weeks. The interns are broken into groups of between four and six people, and each group is a team that works together for the duration of the program. The team has a team leader, and this leader rotates every two weeks so that each member of the team has at least one opportunity to gain leadership experience. There are seven core areas to the program.

1. AI education – every Tuesday, one of our AI educators provides one hour of hands on instruction on artificial intelligence.
2. Running a business – Wednesdays, a member of the team at all different levels (from associate to managing partner) provides real world guidance on what it means to run a business. They are also providing professional stories, lessons on failure, and an opportunity for questions and answers. We invite speakers and leaders from other companies to join us as well on selected topics.
3. Presentation and demonstration – Thursdays, on a rotating basis, three interns will present and showcase their individual projects and accomplishments. They will get feedback. This gives every individual their chance to learn presentation and demonstration skills. There is no competition at the individual project level.
4. Teamwork and competition – Fridays, three teams present and demonstrate their team project, based on the leader’s guidance and leadership style. They will get feedback and compete as a team for a prize. At the end of the internship program, we will have a showcase showdown “shark tank” style where all teams demonstrate their final project and compete for prizes and awards.
5. Portfolio building – each intern is part of a team, and that team has a group project to complete that is relevant to their teams’ business focus area. The focus area is assigned at the beginning of the program. Each intern also has their individual project that they select based on their interests. Both projects involve building something, which the students will vibe code and package in their portfolio. When the interns leave, they will have a portfolio, and three things in their portfolio – an individual project, a team project, and a case study written by them.
6. Adding value to a business – each intern has a PhoenixTeam manager with whom they work to add value to the business. What the intern works on will be set by that manager, and the manager will meet with the intern regularly on their assignments. Their assignments will contribute to the goals of PhoenixTeam and add value to the organization.
7. Leadership – interns alternate being the leaders of their team and learn what leadership is and how to lead. They will practice setting weekly and program level goals and guiding a team to achieve those goals.

The program culminates in a showcase showdown, where teams compete in front of a “shark tank” style panel. They present their project in seven minutes to the managing partners and demonstrate their product.

Week-by-Week AI Education

Week-by-Week Running a Business

We are really excited about making a real difference for this demographic, we talk about it all the time in our classes. It's time to do something real about it and this is a first step.

Follow us for more updates as the AI Futures Internship Program gets underway.

By Tela Mathias, CTO and Chief Nerd and Mad Scientist at PhoenixTeam

‍

‍

ARLINGTON, VA, UNITED STATES, April 28, 2026 -- PhoenixTeam today announced that both PhoenixTeam and Phoenix Burst, its generative AI platform, have achieved SOC 2 Type II compliance, marking an important step in the company’s continued focus on security, trust, and enterprise readiness. This milestone reflects PhoenixTeam’s commitment to building AI solutions that organizations can adopt with confidence.

For mortgage lenders, servicers, financial institutions, and federal housing stakeholders, trust is not optional. As AI becomes more embedded in compliance, operations, and technology workflows, organizations need solutions that help teams move faster without compromising the standards required in regulated environments. Phoenix Burst was built with that reality in mind.

“We built Phoenix Burst for environments where accuracy, accountability, and trust cannot be treated as afterthoughts,” said Tela Mathias, CTO of PhoenixTeam and CEO of Phoenix Burst. “SOC 2 Type II reflects the discipline behind our platform and the seriousness we bring to helping clients adopt AI in workflows that carry real operational and regulatory consequences.”

Established by the American Institute of Certified Public Accountants (AICPA), SOC 2 is a widely recognized standard for controls related to security, availability, processing integrity, confidentiality, and privacy. With support from Johanson Group LLP, a premier certification body specializing in SOC 2 audits across industries, the assessment evaluated how effectively PhoenixTeam’s controls operated over time against the AICPA’s Trust Services Criteria.

This achievement reflects PhoenixTeam’s broader commitment to building technology and AI solutions that are ready for real-world use in high-stakes environments. As organizations move beyond experimentation and toward operational adoption, PhoenixTeam remains focused on helping clients implement AI in ways that are practical, responsible, and built to hold up in production. SOC 2 Type II adds another layer of confidence for teams using Phoenix Burst to streamline execution, reduce friction, and modernize with greater trust.

About PhoenixTeam

PhoenixTeam is a woman-owned technology services firm headquartered in Arlington, Virginia, specializing in AI-powered mortgage operations and technology services for the mortgage and financial services industries and federal housing agencies. Our mission is to enable affordable and accessible homeownership through innovative, customer-centric technology. With a strong focus on generative AI, we tackle complex industry challenges, equipping businesses with cutting-edge tools that enhance innovation, efficiency, and compliance. By bridging the gap between technology and business teams, we strive to bring joy and purpose back to software development, making a meaningful impact in the lives of our clients and homeowners everywhere. For more information, please visit www.phoenixoutcomes.com.

‍

What if we've been looking at technical debt all wrong? There is a page in the Thinking Machine that talks about how the engineers from a company Jensen acquired got under the hood with the Nvidia code base. They were horrified. It was an obvious compromise, filled with workarounds, quick fixes, hackey solutions. It was a code base latent with technical debt. Why? Because technical debt is a spoil earned by winners.

While this acquired company had been perfecting its code base, focusing on engineering perfection, Nvidia had been, well, winning. And their code base suffered. I think we see this today as well with some of the really successful companies - look at the Anthropic Claude Code code leak. All the criticism. For what? We should all be so lucky as to have a team that pushes as many valuable products and features as Anthropic does.

So what if our goal should be to incur as much technical debt as possible? Consider the following continuous innovation flow:

1. Take the people with the actual problems - mortgage operators - and provide them with a deep experiential understanding of generative artificial intelligence. This includes all the foundations, how it all works, the risks, the rewards, and everything in between. They will come out of this process forever changed.
2. Give them the tools they need to fix their own problems. This includes big-ass developer boxes, the Claude suite of products, and hands-on education on these solutions necessary to build great solutions. They know the problems they need to solve - this is the huge time save. You don't have to teach them the mortgage domain, they already know it! You do, however, have to teach them to generate (or give them access to) a high quality synthetic test data set for their use cases. Great news here as well, they know the data, and the tools you gave them are excellent at generating synthetic test data, another win.
3. Provision role-based, controlled access to the data they need for those solutions as MCP servers. The organization partners with the operators to understand what data they need, and to provision that data as MCP servers, removing the technical implementation specifics from the operators. Obviously, this is the long pole in getting the innovation pipeline up and running. The mortgage operators with the problems will have a "need to know" the data in question, and you will control the access, so there should not be a problem giving them controlled access to production data. Also keep in mind there will be a ton of use cases for which borrower data is simple not required. Yes we have to be cautious here, and also we don't have to be ridiculous about it.
4. Provide an automated commit process controlled by the operators that moves the locally developed solutions to the organizational github repository. This step kicks of the automated safety review.
5. Perform a thorough, automated safety review. Yes, you can have a human in the loop here if you want. You can insert a chief software engineer here, but wouldn't it be better to just run an automated review? All the security scans and safety protocols you need can likely be automated. So automate them. Keep in mind - you have given them authorized tools and authorized datasets, and they are going to deploy to an internal innovation site. You have already systematically controlled for safety.
6. For solutions that need access to highly sensitive data, the successful completion of this will move the solution to an integrated pre-production environment for testing of these solutions by their inventors with production data. Again, this will not be every solution, but some of these solutions will have to go through pre-prod review as well.
7. Once the safety review and pre-production review are complete, we automatically (or with a human in the loop) move these solutions to a role-based, access controlled internal innovation site. The inventors know who needs access, and those individuals can submit a simple request for access. That access review can be automated, or you can keep a human in the loop. You already provisioned the data here, and you tested the applications for safety. You know who can have access to what data, this can be automated.
8. Now we can harness the value and the learnings, showcase the solutions. The owness is on the inventors to capture the value and document the learnings. We create an easy way for this to be captured, a simple wiki perhaps, and a leaderboard. Who found the most value? What is the best learning? These inventors also make up your innovation working group so they are already meeting and sharing learnings. They will operationalize learning, they will find new problems to solve based on these learnings. They will create the exuberance from their teams and leaders about the solutions through the (weekly? monthly?) innovation showcase.
9. And finally, we will have solutions that we also want to move selected solutions to our enterprise solution base. Identify those solutions and put them into the regular (but accelerated with AI-assisted software engineering) process for enterprise release. Move as quickly as responsibly possible to enhance the core systems, but with less pressure. The inventors have already solved the problem, you've bought yourself a little time.

And the cycle continues. More data is safely provisioned, more learning are learned, more value is created, more wins are showcased. Yes, more technical debt will have been created and will have to be supported and maintained. To the victors, go the spoils.

Yes, I know this is an uncomfortable idea. Yes, we are heavily regulated and we do have to consider access to production data. Yes, this process will have to be battle-tested and tailored for your organization. But, again, these are mortgage operators, they already have access to this information, they know the problems because they are living with them every day. They are already working around these problems every day. Give them the power to solve those problems.

This continuous mortgage innovation pipeline puts the whole process on its head, it skips numerous steps in the product development lifecycle, not the least of which is teaching software engineers about mortgage. Let's face it, this is one of the most delay-laden steps in our current ways of working. So eliminate that step.

By Tela Gallagher Mathias, CTO and Chief Nerd and Mad Scientest at PhoenixTeam