System Integration Testing: And Whose Bug It Is

A payment fails in the shared test environment on a Wednesday afternoon. The payments team looks at their logs and sees a malformed response from the ledger. The ledger team looks at theirs and sees a request missing a field that has been required since March. Both teams are certain the other one is wrong.

Four days later the two leads meet, read the API specification together, and discover it says nothing about that field at all. The specification was written in January. Nobody updated it. System integration testing is the phase where this happens, and every article on the topic explains the phase while skipping the part that actually costs money. This post is about the seam, and about who owns a failure found on it.

What system integration testing is

System integration testing checks that separately built systems work together once they are connected, using their real interfaces.

The emphasis is on separately built. Integration testing inside one codebase checks that your modules agree. System integration testing checks that your system and somebody else's system agree, where the other side has its own team, its own release schedule and its own idea of what the contract says.

That is usually where the interesting failures live. Two teams can each hold a correct implementation of two different understandings.

Where it sits

The layers either side of it are easy to confuse, so it is worth being exact.

• Integration testing connects components you own, usually in one repository, often with the far side stubbed
• System integration testing connects whole systems across an ownership boundary, with nothing stubbed on either side
• End-to-end testing runs a full user journey through all of it, and cares about the outcome rather than the interface

The practical difference is who you have to talk to when it goes red. An integration failure is a conversation with a colleague. A system integration failure is a conversation with another team, another vendor, or a bank.

The ownership problem

This is the part that gets skipped, and it is the reason SIT phases overrun.

When a test fails inside one team's boundary, ownership is obvious. When it fails at a seam, there are three possible owners and no default:

• The caller, for sending something the contract did not allow
• The receiver, for rejecting something the contract did allow
• The contract, for not saying

The third is the most common and the least often considered. In the Wednesday example above, nobody was wrong. The specification was silent, both teams filled the silence differently, and four days went into establishing that.

The fix is a rule agreed before the phase starts, written down where both teams can see it. The rule that works in most organisations is simple. A failure at the seam belongs to the contract owner until the contract is shown to be unambiguous. Only then does it move to whichever side broke it.

That inverts the usual instinct, which is to argue about implementations first. Reading the contract first is faster, because roughly half the time the contract does not answer the question and the argument was never going to resolve.

The contract case that settles it

A dispute at a seam is settled by a document, not by a conversation. The document is a test case that says precisely what each side sends and returns.

Most teams have an API specification, and a specification is not enough on its own. A specification says a field is a string. It does not say what happens when the string is empty, when it is 400 characters, or when it arrives with a trailing space. Those are exactly the cases the two teams will disagree about.

A contract case pins the behaviour down with a real example on both sides. That is the same discipline that makes an API test case worth having, applied across an ownership boundary rather than inside one.

The set worth writing for any seam is small:

• The normal request and its exact expected response
• The request with each optional field omitted
• The request with a field at its documented limit, and one past it
• The behaviour when the receiver is slow, and when it is unavailable
• The behaviour on a duplicate, which is where retries bite

The last two are the ones teams skip and then argue about at four in the afternoon. Negative cases are where the disagreements actually live, because the happy path is the part both teams already agreed on.

The environment problem

The other reason SIT phases slip has nothing to do with disagreement. It is that the phase needs several systems available at once, and nobody controls all of them.

A typical seam involves your service, another team's service, a third party sandbox, and a shared database. On any given Tuesday one of those four is redeploying, rate limiting you, or has been reset overnight by somebody who needed clean data.

Three things reduce the pain, and none of them eliminate it:

• Book the environment. A shared calendar for a shared resource sounds bureaucratic until you lose a day to a silent redeploy.
• Record which versions were in play when a case ran. A pass against last week's build of the other side is not a pass.
• Keep a small set of cases runnable against a stubbed far side, so you can tell a genuine contract failure from an unavailable dependency.

That third point is worth expanding. The purpose of SIT is to run against the real far side, so stubs seem to defeat the point. They do not, if you use them for triage rather than for verification. When a case fails against the real system and passes against a stub built from the contract, you have learned something specific. The other side is not behaving as documented, and the conversation starts from evidence.

A written case

Here is one, for a payments product calling an external ledger.

Title: A duplicate transfer reference is rejected by the ledger

Preconditions: The payments service and ledger service are both deployed, with versions recorded. A transfer with reference TR-88421 has already been accepted. The test account balance is recorded before the case runs.

Steps:

1. Send a transfer request with reference TR-88421 and amount 250.00
2. Record the HTTP status and response body
3. Query the ledger for entries matching that reference

Expected result: The ledger returns 409 Conflict with an error code of duplicate_reference. Exactly one ledger entry exists for TR-88421. The account balance is unchanged from the recorded value. The payments service records the rejection without retrying.

That last line is the one that matters at a seam. A caller that retries a 409 turns a correctly rejected duplicate into an outage, and neither team's own test suite would show it.

Where it fits

System integration testing is the phase most likely to slip, and the reasons are organisational rather than technical. Two teams, one contract, and a shared environment nobody owns.

The wider map of testing types covers how the layers relate. The specific advice for this one is narrower. Agree the ownership rule before the phase starts, write the contract as cases rather than as prose, and record which versions were running when each case passed. Keeping that record is ordinary test case management work, and it is what turns a four day argument into a four minute one.