{"id":50,"date":"2026-08-14T13:08:25","date_gmt":"2026-08-14T13:08:25","guid":{"rendered":"https:\/\/barrymind.com\/?p=50"},"modified":"2026-08-14T13:08:28","modified_gmt":"2026-08-14T13:08:28","slug":"when-encompass-says-not-now","status":"publish","type":"post","link":"https:\/\/barrymind.com\/?p=50","title":{"rendered":"When Encompass Says \u201cNot Now\u201d"},"content":{"rendered":"\n

Building an Asynchronous Retry System<\/h2>\n\n\n\n

There is a particular kind of software problem that doesn’t look very complicated at first.<\/p>\n\n\n\n

You save something.<\/p>\n\n\n\n

The system says no.<\/p>\n\n\n\n

You try again.<\/p>\n\n\n\n

It says no again.<\/p>\n\n\n\n

You wait.<\/p>\n\n\n\n

You try again.<\/p>\n\n\n\n

Still no.<\/p>\n\n\n\n

Eventually, you realize the problem isn’t really your code.<\/p>\n\n\n\n

Someone has the loan locked.<\/strong><\/p>\n\n\n\n

Welcome to one of the realities of building software around Encompass.<\/p>\n\n\n\n

The Problem: You Don’t Always Own the Loan<\/h2>\n\n\n\n

When working with Encompass, modifying a loan generally requires the loan to be locked.<\/p>\n\n\n\n

That makes perfect sense from a data consistency perspective. You don’t want two different processes changing the same loan at the same time and creating a beautiful little data disaster.<\/p>\n\n\n\n

The problem is that a loan can remain locked for quite a while.<\/p>\n\n\n\n

A broker might have the loan open in the Encompass client. They might be reviewing it, editing something, or simply have it open while they go get coffee.<\/p>\n\n\n\n

Meanwhile, our application is trying to save something.<\/p>\n\n\n\n

And Encompass basically says:<\/p>\n\n\n\n

\n

Nope. This loan is busy.<\/strong><\/p>\n<\/blockquote>\n\n\n\n

From the broker’s perspective, this is even worse.<\/p>\n\n\n\n

They click Save<\/strong> on our application, and the operation fails because somebody else happens to be using the loan.<\/p>\n\n\n\n

That isn’t a particularly friendly user experience.<\/p>\n\n\n\n

And there was another problem.<\/p>\n\n\n\n

SDK + Speed = A Dangerous Combination<\/h2>\n\n\n\n

Anyone who has spent enough time working with the Encompass SDK knows that it isn’t exactly famous for being fast.<\/p>\n\n\n\n

Calling the SDK can be surprisingly slow.<\/p>\n\n\n\n

And when you put a slow external operation directly into a web request, the problem isn’t just that the operation takes a long time.<\/p>\n\n\n\n

Your application has to wait for it.<\/strong><\/p>\n\n\n\n

The user has to wait.<\/p>\n\n\n\n

The web server has to keep the request alive.<\/p>\n\n\n\n

Threads remain occupied.<\/p>\n\n\n\n

And if you have a lot of users doing this at the same time, a slow Encompass operation can start affecting the performance of the entire application.<\/p>\n\n\n\n

So now we had two problems:<\/p>\n\n\n\n

    \n
  1. The loan might be locked.<\/li>\n\n\n\n
  2. Even when it isn’t locked, the operation itself may take a while.<\/li>\n<\/ol>\n\n\n\n

    Putting all of that directly into a web request is not exactly a recipe for a relaxing afternoon.<\/p>\n\n\n\n

    We needed a different approach.<\/p>\n\n\n\n

    So We Built a Retry Machine<\/h2>\n\n\n\n

    Instead of making every Encompass update happen directly inside the user’s request, we built our own asynchronous retry mechanism.<\/p>\n\n\n\n

    I like to think of it as a retry machine<\/strong>.<\/p>\n\n\n\n

    The basic idea is simple:<\/p>\n\n\n\n

    \n

    Don’t make the user fight Encompass. Put the work in a queue and let the system fight Encompass instead.<\/strong><\/p>\n<\/blockquote>\n\n\n\n

    Every operation that needs to update an Encompass loan goes through the retry system.<\/p>\n\n\n\n

    That includes things like:<\/p>\n\n\n\n

      \n
    • Updating loan data<\/li>\n\n\n\n
    • Uploading documents<\/li>\n\n\n\n
    • Other Encompass operations that may require the loan to be available<\/li>\n<\/ul>\n\n\n\n

      When we create an operation, we don’t just store a few parameters and hope for the best.<\/p>\n\n\n\n

      We store the information required to reproduce the operation, including the relevant data and the logic needed to execute it.<\/p>\n\n\n\n

      The request is then placed into a queue.<\/p>\n\n\n\n

      From there, the retry system processes the requests in order.<\/p>\n\n\n\n

      Something roughly like this:<\/p>\n\n\n\n

      Broker\n   \u2502\n   \u2502 Save\n   \u25bc\nOur Application\n   \u2502\n   \u25bc\nRetry Queue\n   \u2502\n   \u251c\u2500\u2500 Operation #101\n   \u251c\u2500\u2500 Operation #102\n   \u251c\u2500\u2500 Operation #103\n   \u2514\u2500\u2500 Operation #104\n          \u2502\n          \u25bc\n      Encompass\n<\/code><\/pre>\n\n\n\n
      If Encompass accepts the operation, great.<\/p>\n\n\n\n
      We’re done.<\/p>\n\n\n\n
      If Encompass doesn’t accept it because the loan is currently locked, we don’t immediately tell the broker:<\/p>\n\n\n\n
      \n
      “Congratulations! Please try again later.”<\/p>\n<\/blockquote>\n\n\n\n
      Instead, the operation stays in the system.<\/p>\n\n\n\n
      After a configurable delay, we try again.<\/p>\n\n\n\n
      And again, if necessary.<\/p>\n\n\n\n
      The important part is that the user doesn’t have to keep clicking Save<\/strong>.<\/p>\n\n\n\n
      The system does the waiting.<\/p>\n\n\n\n
      The Unexpected Bonus: It Made the Application Faster<\/h2>\n\n\n\n
      There was another benefit that turned out to be just as important.<\/p>\n\n\n\n
      The retry mechanism is asynchronous.<\/strong><\/p>\n\n\n\n
      That means the web request doesn’t have to sit there waiting for Encompass to finish its work.<\/p>\n\n\n\n
      The application can accept the user’s request, put the operation into the queue, and return control to the user.<\/p>\n\n\n\n
      The actual Encompass update happens in the background.<\/p>\n\n\n\n
      This is particularly valuable when working with the Encompass SDK, where some operations can take a surprisingly long time.<\/p>\n\n\n\n
      Without asynchronous processing, a request might look like this:<\/p>\n\n\n\n
      Browser\n   \u2502\n   \u2502 Save\n   \u25bc\nWeb Server\n   \u2502\n   \u2502\u2500\u2500\u2500\u2500\u2500\u2500 Wait \u2500\u2500\u2500\u2500\u2500\u2500\u2502\n   \u2502                  \u2502\n   \u25bc                  \u25bc\nEncompass SDK     Encompass SDK\n   \u2502                  \u2502\n   \u2502\u2500\u2500\u2500\u2500 Slow \u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2502\n   \u25bc\nResponse\n   \u2502\n   \u25bc\nBrowser\n<\/code><\/pre>\n\n\n\n
      With the retry system, the flow becomes much more efficient:<\/p>\n\n\n\n
      Browser\n   \u2502\n   \u2502 Save\n   \u25bc\nWeb Server\n   \u2502\n   \u25bc\nQueue\n   \u2502\n   \u25bc\nResponse\n   \u2502\n   \u25bc\nBrowser\n\n        ...later...\n\n      Background Worker\n             \u2502\n             \u25bc\n        Encompass SDK\n<\/code><\/pre>\n\n\n\n
      Instead of making the user’s request responsible for the entire Encompass operation, we make the request responsible for accepting the work<\/strong>.<\/p>\n\n\n\n
      The background process becomes responsible for completing the work<\/strong>.<\/p>\n\n\n\n
      This dramatically improves the responsiveness and scalability of the application, especially when many users are performing Encompass operations at the same time.<\/p>\n\n\n\n
      In other words, the retry mechanism wasn’t just a reliability feature.<\/p>\n\n\n\n
      It also became a performance feature.<\/strong><\/p>\n\n\n\n
      And that’s one of those architectural improvements that you don’t always appreciate until you remove it and watch the application suddenly become very unhappy.<\/p>\n\n\n\n
      But Asynchronous Updates Have a Catch<\/h2>\n\n\n\n
      At this point, you might be thinking:<\/p>\n\n\n\n
      \n
      “Great. Just put every loan update into the queue and let it run asynchronously.”<\/p>\n<\/blockquote>\n\n\n\n
      Not so fast.<\/p>\n\n\n\n
      This is where mortgage software gets particularly interesting.<\/p>\n\n\n\n
      Loan data is highly interconnected.<\/strong><\/p>\n\n\n\n
      A seemingly small change can have a cascading impact across the entire loan<\/strong>.<\/p>\n\n\n\n
      For example, imagine a broker changes the loan amount.<\/p>\n\n\n\n
      That single change can affect calculations such as:<\/p>\n\n\n\n
        \n
      • LTV<\/li>\n\n\n\n
      • DTI<\/li>\n\n\n\n
      • Loan pricing<\/li>\n\n\n\n
      • Qualification results<\/li>\n\n\n\n
      • Disclosures<\/li>\n\n\n\n
      • Underwriting data<\/li>\n\n\n\n
      • Other downstream fields and processes<\/li>\n<\/ul>\n\n\n\n
        Or imagine the borrower’s income changes.<\/p>\n\n\n\n
        That can affect DTI and potentially a whole chain of other calculations and business decisions.<\/p>\n\n\n\n
        In our system, we often relied on Encompass to perform some of these calculations. We also had our own calculations in certain areas, but Encompass is a good example of why loan updates aren’t always isolated events.<\/p>\n\n\n\n
        One field changes.<\/p>\n\n\n\n
        Another field changes.<\/p>\n\n\n\n
        That triggers another calculation.<\/p>\n\n\n\n
        That calculation affects another process.<\/p>\n\n\n\n
        Suddenly, the “simple update” isn’t so simple anymore.<\/p>\n\n\n\n
        So blindly making every front-end update asynchronous could introduce a completely different class of problems.<\/p>\n\n\n\n
        You could end up with something like:<\/p>\n\n\n\n
        User changes Loan Amount\n          \u2502\n          \u25bc\n     Queue Update\n          \u2502\n          \u251c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n          \u25bc               \u25bc\n     UI shows new     Encompass\n     Loan Amount      still has old value\n                          \u2502\n                          \u25bc\n                    Old DTI \/ LTV\n<\/code><\/pre>\n\n\n\n
        Now we have two different versions of the loan temporarily living in the system.<\/p>\n\n\n\n
        That’s not something you want to do casually.<\/p>\n\n\n\n
        Fortunately, the Business Process Helped Us<\/h2>\n\n\n\n
        This is one of those cases where the business workflow actually helped solve a technical problem.<\/p>\n\n\n\n
        In our mortgage workflow, once the Initial Disclosure<\/strong> has been issued, a broker can no longer freely modify certain loan data.<\/p>\n\n\n\n
        If something important needs to change, they have to initiate a Change Circumstance<\/strong> process, and the Lender takes over the actual change.<\/p>\n\n\n\n
        That business rule creates an important boundary.<\/p>\n\n\n\n
        It significantly reduces the possibility of users continuously making major changes to a loan while asynchronous operations are still being processed.<\/p>\n\n\n\n
        In other words, the business process provides a form of synchronization that the software alone would otherwise have to enforce.<\/p>\n\n\n\n
        This is a lesson I’ve learned repeatedly in mortgage software:<\/p>\n\n\n\n
        Sometimes the business workflow is part of the architecture.<\/strong><\/p>\n\n\n\n
        You don’t always solve every consistency problem with more code.<\/p>\n\n\n\n
        Sometimes an existing business rule gives you exactly the boundary you need.<\/p>\n\n\n\n
        And in this case, it was extremely helpful.<\/p>\n\n\n\n
        That doesn’t mean the retry mechanism became unnecessary.<\/p>\n\n\n\n
        Quite the opposite.<\/p>\n\n\n\n
        Even with these business controls in place, Encompass operations can still fail because of loan locking, performance, transient errors, unexpected data, or other conditions.<\/p>\n\n\n\n
        We still needed the retry system.<\/p>\n\n\n\n
        We just didn’t have to pretend that every loan field could safely be treated as an independent asynchronous operation.<\/p>\n\n\n\n
        Then We Discovered the Next Problem<\/h2>\n\n\n\n
        At this point, we had solved the original problem.<\/p>\n\n\n\n
        Almost.<\/p>\n\n\n\n
        Remember what happens when the broker clicks Save?<\/p>\n\n\n\n
        Their changes enter our retry queue.<\/p>\n\n\n\n
        But what if the loan is locked?<\/p>\n\n\n\n
        The update hasn’t actually made it into Encompass yet.<\/p>\n\n\n\n
        The broker, however, doesn’t care about our queue.<\/p>\n\n\n\n
        They expect to see the change they just made.<\/p>\n\n\n\n
        And they’re absolutely right.<\/p>\n\n\n\n
        Imagine this:<\/p>\n\n\n\n
        The broker changes the borrower’s phone number.<\/p>\n\n\n\n
        Our application accepts the change.<\/p>\n\n\n\n
        The update gets queued.<\/p>\n\n\n\n
        Encompass says:<\/p>\n\n\n\n
        “Loan is locked.”<\/strong><\/p>\n\n\n\n
        So the update waits.<\/p>\n\n\n\n
        Then the broker refreshes the page.<\/p>\n\n\n\n
        We read the loan from Encompass.<\/p>\n\n\n\n
        Encompass still contains the old phone number.<\/p>\n\n\n\n
        And suddenly our application appears to have forgotten what the broker just did.<\/p>\n\n\n\n
        That’s not acceptable.<\/p>\n\n\n\n
        So we needed another layer.<\/p>\n\n\n\n
        The Cache: Because Reality Has a Delay<\/h2>\n\n\n\n
        We introduced a cache specifically for pending changes.<\/p>\n\n\n\n
        When a broker changes something, we store the updated value in our cache while the Encompass operation is waiting to be completed.<\/p>\n\n\n\n
        When our application reads the loan, we check the cache first.<\/p>\n\n\n\n
        If the cache contains a newer value, we use that value.<\/p>\n\n\n\n
        Once the Encompass update succeeds, the cached value can be reconciled with the source system.<\/p>\n\n\n\n
        Conceptually:<\/p>\n\n\n\n
                        Read Loan\n                    \u2502\n                    \u25bc\n             \u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\n             \u2502 Cache has    \u2502\n             \u2502 newer data?  \u2502\n             \u2514\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n                Yes  \u2502  No\n                 \u2502   \u2502\n                 \u25bc   \u25bc\n              Cache Encompass\n                 \u2502   \u2502\n                 \u2514\u2500\u2500\u2500\u2518\n                    \u2502\n                    \u25bc\n                   UI\n<\/code><\/pre>\n\n\n\n
        So even if Encompass hasn’t been updated yet, the broker can still see the change they just made.<\/p>\n\n\n\n
        This creates a much better user experience.<\/p>\n\n\n\n
        But, naturally…<\/p>\n\n\n\n
        we had just created another problem.<\/strong><\/p>\n\n\n\n
        Cached Data Has an Expiration Date<\/h2>\n\n\n\n
        A cache is great when it contains newer information than the source system.<\/p>\n\n\n\n
        A cache is dangerous when nobody remembers to update it.<\/p>\n\n\n\n
        Imagine an Encompass operation fails.<\/p>\n\n\n\n
        Then it fails again.<\/p>\n\n\n\n
        And again.<\/p>\n\n\n\n
        Maybe the loan stays locked.<\/p>\n\n\n\n
        Fine.<\/p>\n\n\n\n
        But what if it wasn’t actually a lock?<\/p>\n\n\n\n
        What if the failure is caused by a bug in our code?<\/p>\n\n\n\n
        What if the data is malformed?<\/p>\n\n\n\n
        What if an unexpected value causes the Encompass operation to fail every single time?<\/p>\n\n\n\n
        Now we have a pending update sitting in our queue.<\/p>\n\n\n\n
        And we have cached data that is newer than Encompass.<\/p>\n\n\n\n
        If nobody handles the failed operation, that cached data can sit there for a very long time.<\/p>\n\n\n\n
        Eventually, our “temporary” cache becomes an alternative version of reality.<\/p>\n\n\n\n
        That’s when things get dangerous.<\/p>\n\n\n\n
        Someone Has to Watch the Queue<\/h2>\n\n\n\n
        This is one of the less glamorous parts of building reliable software.<\/p>\n\n\n\n
        Automation is great.<\/p>\n\n\n\n
        But automation doesn’t eliminate the need for humans.<\/p>\n\n\n\n
        It changes the kind of work humans need to do.<\/p>\n\n\n\n
        Our administrators need to monitor failed operations and determine why<\/strong> they failed.<\/p>\n\n\n\n
        If the loan was simply locked, the operation may just need another retry.<\/p>\n\n\n\n
        If the failure is caused by a code bug, retrying it 50 times isn’t going to magically fix the bug.<\/p>\n\n\n\n
        If the data itself is invalid, someone needs to investigate it.<\/p>\n\n\n\n
        And if the system has entered an unexpected state, someone needs to understand what happened before we make things worse.<\/p>\n\n\n\n
        So the retry queue became more than just a technical component.<\/p>\n\n\n\n
        It became an operational tool.<\/p>\n\n\n\n
        We needed:<\/p>\n\n\n\n