Tokyo Stock Exchange made hardware failover a market-continuity accountability test

Summary

• At 07:04 Japan Standard Time on 1 October 2020, arrowhead detected a large volume of access-error messages from memory in one controller of a dual-node network-attached storage device. Equipment failure was the trigger. A component can fail within service life; redundancy existed precisely to contain that event.
• The technical root cause was that the NAS On Panic parameter was set to False under an obsolete understanding that standard takeover would still occur. In the actual product specification, a panic message arriving before heartbeat loss meant that neither immediate nor standard takeover started. A stale manual, limited public evidence specification-change control and unrepresentative failure testing produced that configuration.
• The storage pair held shared files used by market-data distribution, supervisory screens and all three ordinary or emergency trading-suspension paths. This common dependency was a contributing condition: multiple named controls existed, but each waited on the same unavailable NAS.
• TSE detected symptoms quickly, formed an emergency headquarters and worked with Fujitsu. Yet its diagnostic tool also stalled, pre-prepared forced-takeover commands were inadequate, and several attempts failed before a modified command switched control to NAS No. 2 at 09:26.
• Order acceptance had begun at 08:00. Because TSE could not stop execution normally, it isolated arrowhead and ToSTNeT at network load balancers at 08:54 and 08:56. Internal execution processing continued for accepted auction orders, and FLEX disseminated approximately one minute of execution data after 09:00. TSE declared that auction data invalid; it separately confirmed that qualifying ToSTNeT executions were valid. Market-data integrity therefore required legal and operational classification, not simply reconnecting a cable.
• Restarting without a reboot would have released retained execution notices that TSE had already declared invalid. Rebooting could clear exchange-side orders and executions, but entities would not receive ordinary expiry messages and many could not reconstruct or resend customer orders. TSE's consultation indicated that only about 38% of trading value, limited by entity type to overseas securities firms, could be ready. The independent-director committee found the full-day suspension decision reasonable in the circumstances.
• That same committee and the Financial Services Agency separated the reasonableness of the response from inadequacy before the event. TSE had not tested reboot-and-resume even in a test environment, agreed order treatment with entities, or established a clear same-day restart process. A defensible decision not to reopen did not erase the continuity-control failure that made safe reopening impracticable.
• Fujitsu supplied the OEM storage under its brand, wrote or conveyed the product documentation, designed the setting in detailed documents, developed and maintained arrowhead, and supported the incident. Fujitsu publicly accepted shipping-quality responsibility and limited public evidence testing. The investigation placed substantial responsibility for the manual error and setting chain on Fujitsu while also finding that TSE should, to some extent, have required failure testing commensurate with the NAS's importance.
• FSA issued business-improvement orders to both TSE and JPX under separate Financial Instruments and Exchange Act provisions. It required specification-change controls, dependency review, NAS-independent suspension, entity rules and drills, group-wide inspection and a shift from a predominantly “Never Stop” approach toward resilience.
• Management consequences were explicit but procedurally bounded: TSE President Miyahara Koichiro resigned; JPX and TSE executives received pay reductions or formal warnings; Fujitsu reduced five executives' pay and warned three senior officers. These were company governance actions following regulatory orders, not court judgments of damages, negligence or criminal liability.
• Repair proof exists in layers: motherboard replacement and monitored trading on 2 October; an actual-equipment test followed by On Panic=True in production on 4 October; a complete NAS setting review; forced-switch procedures; a NAS-independent emergency halt path; entity drills; published restart, order and information rules; and later arrowhead resilience work. These artifacts materially improve assurance, but no public source proves that every possible hardware state, entity system or future failure can recover without disruption.

Evidence boundary and procedural posture

This analysis uses five evidence labels. A confirmed fact is directly stated in a contemporaneous exchange notice, technical report, company disclosure, rule record or regulatory action. A supported inference connects confirmed facts for control analysis without claiming adjudication. A disputed claim requires materially conflicting positions; mere incompleteness or later refinement is not a dispute. An unknown is a question the public record does not resolve. A legal or regulatory finding is attributed only to the body that made it and only within that proceeding.

The principal forensic record is JPX's independent-outside-director committee report. Its English translation is unusually explicit about its own limit: the committee was composed of JPX outside directors, was not a conventional third-party committee, relied on materials and information provided by JPX and TSE for the factual chronology, and did not make proactive fact-finding or legal responsibility its main objective. It evaluated the companies' account, investigation methods, response and proposed remedies, including Fujitsu's investigation. That is a strong formal governance record, but not an independent judicial finding.

The FSA's Japanese administrative disposition has a different posture. The regulator requested reports on 2 October, received them on 16 October and conducted on-site inspections before issuing business-improvement orders on 30 November. Its findings therefore carry regulatory authority. They do not decide private compensation, allocate contractual damages among TSE, Fujitsu and the unnamed OEM supplier, or establish criminal conduct.

Fujitsu's disclosures are admissions and descriptions by a directly involved vendor. On 19 October it said that the ETERNUS NR1000 was supplied by another developer-manufacturer as an OEM product but that Fujitsu bore responsibility for shipping quality. It acknowledged that the manual did not track an operating-system-driven specification change and that its testing and confirmation were limited public evidence. Those statements are important first-party evidence; they are not an external audit of every Fujitsu or supplier engineering record.

The causal account was refined, not materially disputed. TSE's 1 and 5 October communications described a memory-module failure and failed switchover. The later investigation distinguished the component event, the configuration mechanism and the operational conditions that expanded impact. No cited JPX, TSE, FSA or Fujitsu source formally contests that chain. Claims that the day-long outage was “just a random hardware failure,” or conversely that there was no real hardware failure, are inconsistent with the formal record rather than preserved institutional disputes.

Chronology before allocation

Before 1 October: a high-speed system with inherited assumptions

Arrowhead had operated since 2010. The third generation went live on 5 November 2019 after a hardware refresh and performance upgrade. TSE and Fujitsu's joint launch notice emphasized functionality, stable performance and a safe and secure market. The investigation later described a system of hundreds of synchronized servers that accepted orders, matched them, returned execution notices, distributed market information, interfaced with clearing, and provided supervision and control functions. Its advertised order-response time was about 200 microseconds.

That speed context matters because continuity is not only about preserving server availability. An exchange must preserve a common state across orders, executions, official notifications, public price information, supervision, entity books and eventual clearing. Fast matching increases the amount of state that can diverge in seconds when one control plane is impaired.

The NAS pair did not hold the core synchronized order book as a substitute for arrowhead's matching architecture. It held common files such as issue and user information that multiple servers and operational functions needed. Centralizing those files supported consistency and efficient design. It also meant that market-data and control processes could wait on the same storage service if failover failed.

The key setting predated the third generation. Under the older product behavior, On Panic=False disabled immediate takeover but standard takeover still followed heartbeat loss. That setting worked in the first and second generations. During later product evolution, the NAS specification changed: if a panic notification arrived before heartbeat messages stopped, False also prevented standard takeover. The default changed from False to True, but the relevant written specification did not accurately describe the behavioral change. The older False value was carried into the third generation.

This was not a case in which TSE specified no recovery objective. Its third-generation requirements called for NAS-related functions to switch and continue within 30 seconds after equipment failure. The failure lay between requirement, product behavior, detailed configuration, review and representative test. A paper requirement for continuity did not prove that the deployed standby would meet it.

Prior incidents supplied warning context without proving repetition of the same defect. In 2012, part of a market-information gateway failed and 241 issues were suspended for a morning; the investigation attributed that event to incomplete software switching and limited public evidence switch confirmation. In 2018, a burst of messages from a co-location user overloaded routing equipment and disconnected several entity gateways. TSE's 2018 incident report documented network controls, entity communication and expanded failure testing. Those repairs addressed those mechanisms. They did not test the precise panic-before-heartbeat state that occurred in 2020.

07:04-08:36: detection, diagnosis and the decision to halt

At 07:04, monitoring generated many access-irregularity messages for NAS No. 1. At 07:10, some trading-supervision and operations-management terminals could not log in; other terminals still could. The mixed state complicated initial assessment. The diagnostic tool that should have reported system status remained running without returning a result. TSE informed Fujitsu at 07:10, and Fujitsu platform and application personnel began joint impact isolation at 07:11.

From 07:30, TSE detected delayed issue information and found that scheduled routing-maintenance and communication-start messages had not been sent. CIO Yokoyama Ryusuke headed a system-failure emergency headquarters from 07:37. At 07:55, Fujitsu reported that the messages indicated the NAS No. 1 control mechanism had failed and that the expected switch to NAS No. 2 had not occurred.

At 08:00, arrowhead began accepting orders on the normal timetable. This decision became important later, but it should not be rewritten with hindsight. The committee found that before 08:00 TSE did not yet understand the full event or impact; some screens had been available; and staff believed that normal or emergency suspension controls could still protect the market if needed. It nevertheless later criticized the deeper “Never Stop” assumption that made order acceptance at 08:00 and opening at 09:00 appear absolute instead of conditional on evidence.

At 08:01, TSE notified entities that communication-start messages could not be sent. Recovery and impact analysis continued in parallel. TSE chose 08:30 as the decision threshold for suspending all issues if market-data distribution had not recovered, leaving time to notify entities before the 09:00 opening. Distribution remained impaired. At 08:36, TSE notified entities of suspension under Rule 29, Item 4 of its Business Regulations. The public first English market notice said a technical glitch affected market-data distribution, all listed issues would be halted and order acceptance would be shut down.

The decision instruction and the actual technical stop were different events. Normal trading suspension required the trading-supervision screen; market suspension required the operations-management screen; the emergency job avoided that screen but still read NAS files. All were unavailable. TSE therefore used a network control designed to restrain orders to specific servers: it cut the arrowhead load balancer at 08:54 and ToSTNeT's at 08:56. This prevented new entity communication, but it did not issue a normal instruction to stop matching.

08:16-09:26: failed takeover attempts and state divergence

In parallel, TSE and Fujitsu tried to force NAS No. 2 to take control. Orders to power off NAS No. 1 at 08:16 and transfer control at 08:26 failed. A port block at 08:28 interrupted communication, but NAS No. 1 had already gone down and takeover did not occur. A changed-condition attempt at 08:42 also failed. At 09:23, a command supplied by Fujitsu's product group failed; at 09:26, adding options to that command finally forced the switch to NAS No. 2. Market-data processes and supervisory and management screens were then confirmed usable.

The 82 minutes between initial detection and successful forced takeover were not simply repair time for a dead component. They exposed a recovery-control gap. Because engineers assumed automatic takeover would occur, a validated command and state-dependent manual procedure were not ready. The investigation found preparation limited public evidence at Fujitsu and recognized TSE's own inadequate anticipation.

Meanwhile, accepted orders had created a market-integrity problem. Order acceptance began at 08:00, and real-time order data initially flowed. Since the load-balancer action did not stop the matching engine, arrowhead processed orders accepted up to 08:54. The market-data cut-off crossed the 09:00 opening, so FLEX distributed approximately one minute of apparent execution data. At 09:11 TSE told the market that data distributed up to that point were invalid. Its third English report also stated that ToSTNeT had halted and all FLEX data distributed so far were invalid, while Osaka Exchange derivatives were operating normally.

Later classification was more precise. TSE's same-day trading-status notice said no auction-market executions were legally established, even where FLEX had carried apparent executions, and orders received before suspension would not carry to the next day. It separately said ToSTNeT orders received through 08:56 had executed and all execution notices had been sent. “All data invalid” was an urgent market-data warning, not a statement that every off-auction contract was void.

This distinction is central to accountability. A storage failover could restore file access, but it could not by itself decide which messages constituted valid market acts, align entity systems, restore customer balances or recreate fair price formation. Recovery required both technical state and authoritative market state.

09:26-11:45: why a healthy standby did not make a fair market

After the forced switch, TSE considered two routes: reconnect without rebooting, or reboot arrowhead and reopen. Reconnection would release execution notices retained inside arrowhead for orders processed before the network cut. TSE had already declared those auction results invalid. Entities might treat the notices as valid, update customer balances and continue trading on a false premise. The day's base prices had also been affected by internal processing. The committee considered this route incompatible with a fair and impartial market.

Rebooting would reset exchange-side accepted orders, execution data and base prices. But ordinary entity processes were notification-driven. An order sent to TSE normally changed status when the entity received an acceptance, execution or expiry message. A reboot would clear exchange state without sending the ordinary expiry notifications. Brokers would have to move orders out of a “sent” state using exceptional processing, reconcile customer instructions and resend eligible orders. Some pre-funded retail systems would also need to prevent invalid executions from changing buying power or holdings.

TSE sought views from major domestic brokers, online brokers, overseas firms and shared-system vendors before 11:00. Online brokers said they could not handle the process; a major vendor serving numerous medium-sized entities had never tested it and could not confirm readiness. The committee later quantified the estimated ready population at no more than about 38% by trading-value share and described it, by entity attribute, as overseas securities firms. A reopened market dominated by that subset would exclude much of domestic and retail participation and threaten representative price formation.

At 11:00, JPX's Group CEO and TSE's president joined an extraordinary Risk Management Committee meeting. TSE President Miyahara made the decision to remain closed for the day, announced at 11:45. The fifth same-day TSE report explained that the failed switchover prevented market information, hardware replacement was planned and a daytime reboot could cause investor and entity confusion after consultation with market entities.

11:45-2 October: controlled next-day recovery

TSE did not treat the full-day halt as permission to leave market state ambiguous. Its Japanese notice on the day's handling classified auction and ToSTNeT outcomes. A separate next-day handling notice specified reference prices, continuing short-sale price restrictions, price-limit treatment and special ToSTNeT price arrangements. At 19:25, TSE's resumption notice said auction and ToSTNeT trading were expected to proceed normally on 2 October.

On the hardware side, TSE and Fujitsu replaced the whole motherboard carrying the failed NAS No. 1 memory. The investigation says the system returned to its active-active configuration, rapid manual-switch procedures were available, both organizations established enhanced monitoring, preparatory processing completed normally, and motherboard replacement finished at 22:00. Trading on 2 October completed without problems.

The repair was not yet complete in the assurance sense. On 2 October, the parties located the configuration error. They reproduced the relevant condition on actual equipment, confirmed correct switchover and set On Panic=True in production on 4 October. Operation with the corrected value began on 5 October. TSE's initial English technical notice documented the memory-module failure, failed switch, 4 October setting change and confirmation that automatic switching behaved as expected.

Cause map: trigger, root cause and contributing conditions

Trigger

The trigger was an accidental physical failure that made a memory card in NAS No. 1 unreadable and unwritable. Fujitsu and the unnamed OEM supplier examined the part. Their reported failure history for the same model remained within Fujitsu's internal threshold, the card was within service life and they did not identify a defective production lot. The committee did not fault TSE for the occurrence of that component failure.

Calling the memory failure a trigger is not semantic evasion. A standby design is created because components fail. If one in-service component can stop a market despite installed redundancy, accountability analysis must examine why the containment control failed.

Technical root cause

The technical root cause was the incorrect On Panic=False setting for the deployed NAS behavior. The failed controller sent a panic notification before 0.5-second heartbeat monitoring ceased. Under the actual specification, False suppressed immediate takeover and, in that sequence, also suppressed standard takeover. NAS No. 2 did not seize control, leaving the pair unavailable to dependent functions.

The setting chain had several connected facts. The old product had allowed standard takeover with False. The later specification and default changed. The supplier's document recorded the default change but not the behavioral change; Fujitsu's manual remained inaccurate. Fujitsu wrote False into the detailed design, TSE reviewed that design against the same incorrect manual, and neither organization's pre-production test reproduced loss of NAS function under the panic sequence. TSE's one-page NAS setting supplement diagrammed why the panic message precluded the expected delayed takeover.

Fujitsu's own cause-and-measures disclosure confirmed the operative chain: the manual said automatic switching always occurred, the actual product could be configured not to switch under the memory-failure condition, the manual had not followed a specification change introduced through an OS version update, and Fujitsu's testing and confirmation failed to detect the mismatch.

Contributing condition 1: specification governance did not preserve the requirement

TSE required continued processing within 30 seconds. Fujitsu's system engineers knew the configured values, but Fujitsu told the committee that TSE's requirements were not conveyed to the product group during its consistency review. The product group checked whether equipment settings conflicted with one another, not whether the device behavior met the exchange's system-level recovery objective.

This is a supported governance inference: the requirement, implementation and product-validation loops were not closed. No single document holder had proven that “30-second continuation” remained true after the OEM behavior changed. The inference does not establish that any person knowingly concealed a change.

Contributing condition 2: the test represented heartbeat loss, not the actual failure state

TSE and Fujitsu tested interruption of heartbeat communication and saw service continue. They treated that as sufficient evidence for loss of the NAS controller. It was not equivalent. In the actual event, the controller emitted a panic message before heartbeat loss, selecting a different branch in the takeover logic. A test named “switchover” therefore passed while the production failure mode remained untested.

The control lesson is precise: test coverage must be expressed in triggering states and resulting invariants, not only in broad labels. Evidence that a standby takes over after silent heartbeat loss does not prove takeover after a controller panic, partial I/O failure, stale ownership state or malformed notification.

Contributing condition 3: multiple controls shared one storage dependency

The NAS centralized shared files to preserve internal data consistency. Market-information distribution, supervision screens, operations screens, normal halt, market halt and the emergency halt job all relied on those files. The emergency path was procedurally distinct but not failure-domain independent. When the NAS pair became unavailable, every designed suspension instruction failed.

This dependency expanded a storage incident into an inability to control trading. It also forced network isolation, which stopped communication without stopping matching. The FSA later ordered TSE to inspect dependencies throughout the system and ensure that failure of a specific location could not disable functions essential to continuity.

Contributing condition 4: forced recovery was improvised

Automatic switching was expected, so exact commands for taking control under a failed-switch state were not prepared and validated. Several attempts failed before Fujitsu supplied the successful command options at 09:26. Physical cable disconnection was considered. A standby is not operationally available merely because its hardware is powered and synchronized; the operator needs a state-aware, rehearsed authority-transfer path that works when automation does not.

Contributing condition 5: order acceptance and restart policy were not designed as one continuity system

TSE's policy favored accepting orders during suspensions because entities had requested that behavior after earlier events. On 1 October, accepting orders before impact was fully understood made the later abnormal stop harder to reconcile. There was no agreed rule for when to stop order intake, what would happen to accepted orders after a reboot, which notifications controlled entity state, or how to determine that enough market diversity could return.

The independent committee found TSE's real-time decisions reasonable given what staff knew. It nevertheless found the advance decision criteria, procedures, entity communication and test preparation limited public evidence. These conclusions coexist. Incident command can make the least harmful available decision while governance remains accountable for allowing only harmful recovery options.

Contributing condition 6: reliability culture outweighed recoverability

The committee used unusually direct language about “Never Stop.” It said overemphasis on starting order acceptance at 08:00 and trading at 09:00 contributed to making smooth restart impossible after recovery. FSA likewise found that JPX's resilience work lagged its reliability focus. The supported inference is not that high availability was misguided. It is that prevention of interruption had become a stronger design premise than controlled degradation, authoritative halt, state reset and tested reopening.

Detection, response, recovery and market-data integrity

Detection

Detection was neither absent nor fully effective. Monitoring surfaced NAS access abnormalities immediately at 07:04, and operational symptoms appeared within minutes. TSE escalated to Fujitsu and an emergency headquarters before the order-acceptance window. Those are positive controls.

At the same time, the diagnostic tool itself waited indefinitely, terminal availability was inconsistent, and early market-data omissions had to be assembled from separate signs. Detection revealed distress but did not promptly explain the failure domain or guarantee a safe stop. The residual lesson is to separate observability from the component being observed and to predefine market-opening evidence: required control screens, data feeds, stop paths and entity messages must all pass before accepting orders.

Response

The response had three simultaneous objectives: restore storage, prevent an unfair market from opening, and communicate authoritative status. TSE pursued all three. It attempted forced takeover, set an 08:30 decision threshold, notified entities at 08:36, isolated order connections before the scheduled opening, invalidated misleading auction data, consulted entity types and made a formal full-day decision.

The response also exposed implementation weaknesses. Network cut-off completed close enough to 09:00 for one minute of execution data to escape. The first public notices necessarily evolved, but the sequence left users waiting for a recovery timetable. The later recurrence council's December 2020 information-policy draft proposed a dedicated system-status page, updates ordinarily every 30 minutes even without progress, event-impact-resumption stages and push distribution. That remedy is evidence that communication cadence and structure were part of the continuity gap.

Why not restarting was an integrity control

The lost trading opportunity was severe. It does not follow that any reopening was better than remaining closed. A market is not continuous in the relevant sense if only a narrow entity class can reconstruct orders, if invalid execution notices may enter broker books, or if public price messages conflict with legal trade status.

The full-day decision preserved three forms of integrity. Transaction integrity required a common answer about which orders and trades existed. Data integrity required authoritative treatment of FLEX messages and next-day reference values. Participation integrity required a sufficiently broad set of domestic, retail, institutional and overseas entities for fair price formation. The official evidence supports TSE's conclusion that those conditions could not be proved for a same-day restart.

The committee did not merely defer to management. It assessed the 38% readiness estimate, entity composition, absence of a test-environment restart and message-state problem, then found the decision and information-gathering process reasonable. This is the strongest evidence against the simplistic claim that management chose closure only to avoid operational inconvenience.

Recovery

Recovery occurred in stages. At 09:26, forced takeover restored access and operational screens. By evening, TSE had classified the day's trading state, specified next-day market treatment, replaced the failed motherboard, arranged joint monitoring and completed preparatory processing. Normal trading on 2 October provided an important service-level result.

On 4 October, after reproducing the condition on actual equipment, TSE changed On Panic to True. By 23 October, Fujitsu's system and product teams had reviewed all NAS settings against requirements and actual product behavior; by the end of October, values that differed from shipment defaults had received actual-equipment confirmation. The investigation also reported confirmation of forced-switch procedures and plans for equipment-state-specific command lists.

Those stages should not be collapsed into “fixed overnight.” The component replacement restored capacity; enhanced monitoring managed short-term uncertainty; the setting change removed the known automatic-switch defect; comprehensive review looked for sibling configuration errors; and procedures addressed automation failure. Each answers a different assurance question.

Accountability allocation after chronology

Tokyo Stock Exchange: market operator and system owner

TSE held the primary public-facing control over whether its cash market opened, which orders were accepted, whether messages represented valid trades, when trading stopped and when it could resume. It specified the 30-second NAS continuation requirement, reviewed detailed designs, operated arrowhead, maintained the contingency plan and coordinated connected entities.

Its accountability is therefore broader than authorship of the bad setting. TSE could reasonably rely on a specialist vendor's accurate manual, and the committee placed primary responsibility for that error on Fujitsu. But TSE controlled acceptance criteria for a system whose failure could stop a national market. The committee found that, given the NAS's importance, TSE should to some extent have required actual loss-of-function testing and was partly responsible for not recognizing the setting defect.

TSE also owned the design of market continuity around technical failure. Multiple stop mechanisms shared the failed NAS; reboot-and-resume had not been tested even in a test environment; entities had no agreed order treatment; and resumption criteria were not explicit. Those are exchange controls even where entity and vendor cooperation is necessary.

The full-day decision should not be converted into a separate fault finding. The committee found no decision error in remaining closed. TSE's accountability lies in the pre-event architecture and rules that made a fair same-day reopening unprovable, plus the duty to correct them.

Japan Exchange Group: parent governance and group resilience

JPX owned group risk governance, resource allocation and oversight of TSE. The event affected other cash exchanges that used arrowhead and raised common questions for Osaka Exchange and other group market functions even though OSE derivatives remained open. Parent accountability therefore involved more than supervising one incident team.

FSA ordered JPX to have group companies inspect systems and rehearse early recovery, create entity-inclusive restart rules, and rebalance development and maintenance toward resilience. The independent report likewise called for group-wide reliability and “omnidirectional” communication. JPX's role was to turn one subsidiary's failure into a standard across market businesses, not assume the defective configuration was isolated.

Fujitsu: branded product supplier, system integrator and maintenance vendor

Fujitsu had several controls. It supplied the ETERNUS NR1000 under its brand; its product group validated the OEM product and documentation; its arrowhead engineers wrote the detailed NAS setting; it developed, maintained and serviced the exchange system; and it supplied incident commands. These roles gave Fujitsu practical control over the specification-change chain and the translation from product behavior to system requirement.

The investigation concluded that Fujitsu was responsible for the erroneous manual and bore substantial responsibility for the flawed setting. It also found limited public evidence product-group validation, communication with the OEM supplier, failure-state testing, internal coordination and preparation for manual switching. Fujitsu itself accepted that an OEM relationship did not transfer shipping-quality responsibility.

Fujitsu did not control whether TSE legally opened or closed the market, which orders were valid, or what entity breadth was sufficient for price formation. Nor does the cited record establish the private contractual allocation of damages. Vendor responsibility and exchange responsibility are overlapping controls, not a zero-sum choice.

The unnamed OEM supplier

The report calls the underlying NAS source “Company A.” That supplier changed product behavior and the initial default, while its specification did not adequately capture the behavioral change. It also participated in component examination. These facts support an operational role in product specification and test evidence.

The public record withholds the supplier's identity and contract and does not publish its independent response. It would be unsound to assign a percentage of legal responsibility or infer concealment. Fujitsu's explicit acceptance of branded shipping quality means the OEM relationship did not remove Fujitsu's customer-facing control.

Trading entities and shared-system vendors

Brokers controlled their order-state machines, customer instructions, pre-funding, exception handling and readiness to reconcile and resend. Shared platform vendors controlled whether many entities could execute an exceptional recovery. Their inability to support a reboot was a contributing continuity fact, but the investigation did not frame it as misconduct: TSE had not agreed or tested the procedure with them.

After the event, participation created reciprocal duties. TSE had to publish clear rules and test facilities; entities had to implement and rehearse the agreed handling. A restart rule that exists only inside exchange documentation is not market continuity. Conversely, a entity cannot demand same-day reopening while declining to build the message and customer-state controls that make it fair.

Regulators

FSA did not design arrowhead or operate incident command. Its controls were supervision, reporting demands, inspection and enforcement. The 30 November order documented system, procedural and governance deficiencies and required regular reporting. The regulator's role was to convert a company remediation plan into supervised obligations and to require responsibility to be made explicit.

The event also tested the limits of compliance-by-availability metric. An exchange can report high historical uptime while lacking credible recovery from an abnormal stop. Regulatory accountability should therefore examine tested state reconciliation, independent stop paths, entity coverage and evidence from disruptive exercises, not only nominal redundancy and outage duration.

FSA findings and management accountability

FSA found that a product defect directly caused the event, automatic-switch settings were inadequate, and TSE's restart rules were limited public evidence. For TSE, the order required four connected programs: reconfirm settings and change-control processes, including requirements imposed on outsourced vendors; remove critical trading-suspension dependencies on the failed device and inspect all system dependencies; create effective order-acceptance and restart rules with entity tests; and expand resilience alongside “Never Stop” reliability.

For JPX, FSA required group companies to inspect systems and train for early restoration, establish pre-agreed restart decisions with entities, and improve resilience across development and maintenance. An official English reference translation of the disposition preserves these requirements for non-Japanese readers. The Japanese original controls if translation differs.

JPX's 30 November responsibility disclosure then identified management consequences. TSE President Miyahara Koichiro resigned from TSE and as JPX group Co-COO. JPX CEO Kiyota Akira's monthly pay was reduced 50% for four months, CIO Yokoyama Ryusuke's by 20% for four months, and a TSE executive's by 10% for four months; two department heads received severe warnings.

Fujitsu followed with a 3 December executive-action notice. Its president's monthly remuneration was reduced by 50% for four months, its deputy president's by 30%, and three executive officers' by 20% or 10%; three corporate executive officers received severe warnings. The company linked those measures to the failed automatic switch and began broader system reinspection and quality-assurance strengthening.

These measures make management accountability visible, but they do not prove proportionality or effectiveness. A resignation and pay cut show that boards assigned institutional consequence. They do not replace configuration evidence, independent testing, entity readiness or a regulator's follow-up. Nor are they findings that the named executives personally configured the device or committed a legal wrong.

Repair proof: from configuration correction to a restartable market

Immediate technical proof

The strongest immediate proof was not a policy statement. TSE and Fujitsu reproduced the failure condition on actual equipment, observed correct takeover with the changed setting and then put On Panic=True into production. They reviewed all NAS settings against requirement and product behavior, with actual-equipment checks for non-default values. They also confirmed a forced-switch procedure for cases in which automation still failed.

This evidence directly addresses the known causal path. Its limit is scope: it proves a tested scenario and configuration, not all conceivable controller, network, storage or corruption states. The committee explicitly warned that “unexpected” failures remain possible and that vendors should perform comprehensive scenario tests.

Independent market-stop proof

TSE examined whether one device or function could prevent a trading-suspension instruction and developed an emergency halt route that did not pass through the NAS. This addressed the hidden common dependency that forced load-balancer isolation. The control should be judged by periodic execution evidence, including proof that it stops matching and order acceptance while preserving an auditable record, not merely by the existence of a separate button or job.

Agreed restart and order-state proof

TSE established a Council for Recurrence Prevention Measures with securities firms, investors, system vendors and FSA observers. The official council record preserves three council meetings and nine specialist working-group sessions. This forum converted restart from an internal exchange decision into a market protocol.

The March 2021 final council report defined rules by failure timing and type, order acceptance, restart decision windows, entity consultation, information cadence and exercise. It says a 23 January 2021 user drill tested the new information process and that live arrowhead information operations began in February. That is evidence of implementation, not only intent.

TSE's enduring system-reboot resumption page states the operational consequences. A reboot cancels exchange-side orders; entities generally resend entrusted customer orders unless separately agreed; partially executed orders need linkage; pre-failure valid executions remain valid in principle; restarted FLEX data reflect the post-restart period, with consolidated daily information published later; and price, corporate-information halt and short-selling rules have explicit treatment. These are exactly the state questions that lacked an agreed answer on 1 October.

TSE also opened a 2021 public-comment process for deficiencies in transaction-detail notices caused by system failure. Publishing proposed handling and responses to comments makes the legal-operational boundary reviewable beyond the incident room.

Group-level proof and residual work

JPX's FY2020 management-plan review reported completion of corrected settings and comprehensive checks, forced-switch procedures, key-system checks and a NAS-independent trading-halt function. It described tests and drills as ongoing and the council rules as the basis for continuous effectiveness review. This disclosure is useful management evidence, though it remains first-party status reporting rather than an external technical certification.

The dependency extended beyond TSE's own listings. Nagoya Stock Exchange's 1 October notice stated that its full-day halt arose from the TSE trading system it used. Fukuoka Stock Exchange likewise issued next-day handling instructions after its full-day suspension. Repair therefore had to coordinate the regional exchanges using arrowhead, not only JPX subsidiaries.

Later system evolution provides additional, but not conclusive, continuity evidence. JPX's current arrowhead service history records the fourth generation going live in November 2024 with resilience as an explicit objective, alongside synchronized three-node trading-data processing and current recovery-oriented functions. A successful new generation shows investment and changed design priorities. It cannot retroactively prove that every 2020 weakness was absent at every point after the incident.

Confirmed facts, supported inferences, disputed claims and unknowns

Confirmed facts

The memory card failed; the NAS pair did not switch automatically; On Panic=False had different actual behavior from the manual-based understanding; representative panic-state testing was absent; all ordinary and emergency stop paths depended on NAS files; forced takeover succeeded at 09:26; auction execution data disseminated around 09:00 were invalid; some ToSTNeT transactions remained valid; same-day reboot had not been tested; estimated resend readiness was about 38% by trading value and limited to overseas firms; TSE stayed closed; normal trading completed on 2 October; FSA issued two business-improvement orders; both JPX/TSE and Fujitsu imposed management consequences; and technical and procedural remediation followed.

Supported inferences

Nominal hardware redundancy was not operational redundancy because the deployed state, manual, test and manual-override path did not jointly prove takeover. The shared NAS created a control-plane common mode even though matching data had other synchronization protections. “Never Stop” became a contributing governance condition when opening routines were treated as default despite incomplete readiness evidence. The decision to remain closed protected market integrity, while inability to reopen exposed an avoidable continuity gap. These are control conclusions supported by the record, not legal judgments.

Disputed claims

No material causal dispute among TSE, JPX, FSA and Fujitsu appears in the cited formal record. Early shorthand that attributed the incident to hardware was superseded by, and is incomplete without, the documented setting and governance chain. Public assertions that TSE could simply have switched markets back on are contradicted by the retained notices, entity-state and readiness evidence. Because no formal party presented a tested same-day alternative, this article does not manufacture a two-sided dispute.

Unknowns

The public record does not identify Company A, publish the OEM contract, disclose source code or complete configuration files, provide the failed card's full laboratory report, or show every internal review comment. It does not quantify every investor's economic loss, establish whether TSE or Fujitsu paid private compensation, or adjudicate contractual liability. It does not publish entity-by-entity readiness responses or prove that the 38% estimate was exact. It does not provide complete independent results for every later drill or prove that all regional and entity systems implemented every rule at the same time.

Counterfactual controls and the standard of proof

What the impact record proves and does not prove

The incident's critical impact is confirmed without inventing a yen-loss estimate. TSE provided no auction trading opportunity for the entire scheduled day; ToSTNeT stopped after the pre-open interval; Nagoya and Fukuoka also suspended trading because they used the TSE system; public market data required invalidation and correction; and brokers, issuers and investors had to carry open investment decisions into the next session. The exchange's central price-discovery function was unavailable even though derivatives at Osaka Exchange continued.

The public sources do not provide a complete counterfactual transaction ledger showing what each investor would have bought or sold, at what price, absent the outage. Nor do they aggregate broker remediation cost, hedging slippage in other venues, delayed issuance activity, index effects or private claims. A day of unavailable trading can create opportunity costs and risk exposure, but those outcomes depend on investor intent and later prices. This article therefore does not convert market capitalization, an average day's turnover or an unexecuted order value into damages.

Institutional impact is independently significant. FSA found that the full-day stop seriously damaged investor confidence in a financial instruments exchange, required regular remediation reporting and demanded explicit responsibility. Regional dependence showed that one control failure could affect legally distinct markets. Management actions at JPX/TSE and Fujitsu demonstrated that both boards treated the event as a governance failure, not routine component maintenance. These facts support the CRITICAL impact rating while preserving the difference between loss of a national price-formation service, measurable response costs and unproved private monetary loss.

The narrowest preventive counterfactual is direct: with On Panic=True, the receiving controller would have initiated immediate takeover after the panic notice. Actual-equipment reproduction before the production change supports that conclusion for the known state.

A stronger preventive counterfactual is governance-based. If every product behavior change had been mapped to system requirements, the changed default and panic semantics would have triggered review. If TSE's exact deployed non-default value had been tested under controller loss rather than only heartbeat interruption, the inconsistency could have been detected before service. If the acceptance gate required evidence for each trigger branch, a generic “failover test passed” result would not have closed the requirement.

Impact-reduction controls are separate. A validated forced-takeover runbook could have shortened storage recovery. A NAS-independent command that stopped matching and order intake could have prevented retained execution notices. A market-opening gate could have stopped 08:00 order acceptance once critical control screens and communications were impaired. Pre-agreed reboot semantics and entity drills could have allowed broad, fair resending and same-day recovery.

Proof must be layered. Configuration proof includes signed inventories, requirement-to-setting mappings and actual-device tests. Failure-domain proof includes destructive or fault-injection exercises across panic, heartbeat, partial I/O, control loss and stale ownership states. Market-control proof demonstrates that independent halt paths stop the intended processes and preserve authoritative logs. Restart proof reconciles exchange, broker, customer, public-data and clearing state. Participation proof measures what share and diversity of the market can return inside each recovery window. Governance proof records exceptions, owners, deadlines, board review and regulator follow-up.

Uptime is only one outcome. A resilient exchange must be able to stop safely, explain what is valid, restore from a known state, include a representative market and produce evidence that each step worked. The 2020 outage showed why a standby device, a contingency plan and several emergency functions can all exist while continuity remains unproven.

Conclusion

The 1 October 2020 outage was triggered by a physical memory failure, but its significance came from failed containment. A stale product description, an inherited setting, an untested panic sequence and inadequate manual-switch preparation defeated storage redundancy. Shared dependence then disabled market-data and trading-stop controls. Order acceptance and a network-level cut-off created state that could not be reconciled safely with entity systems under any rehearsed rule.

TSE's choice not to restart in the middle of the day was not evidence of indifference to continuity. On the official record, it was a rational market-integrity decision: reconnection risked delivering invalid executions, rebooting risked excluding most domestic and retail participation, and neither route had been tested. The accountability failure occurred earlier, when the exchange ecosystem had not made safe restart a designed and proved capability.

Responsibility followed control. Fujitsu owned the branded product documentation, setting design, product validation and important maintenance functions, and accepted substantial fault in that chain. TSE owned acceptance criteria, independent halt architecture, market rules and entity readiness. JPX owned group oversight and resilience priorities. Entities owned their order and customer-state recovery once the exchange supplied an agreed protocol. FSA owned supervisory enforcement and required those controls to become explicit.

The repair record is credible because it contains more than apologies: a replaced board, actual-equipment failover reproduction, corrected production setting, setting-wide review, forced-switch procedures, independent halt development, entity exercises, published restart semantics, information cadence and group-level resilience work. Its residual limit is equally important. No finite test proves that a market will never stop. The accountable objective is to show that when an unanticipated component or state fails, the market can halt authoritatively, preserve truth, recover broadly and demonstrate why reopening is fair.