8/16/2023 0 Comments Gemini exchange outage![]() Moving forward and in support of our chaos engineering approach, we will ensure reproducibility of this failure mode in our test environments and improve our trading platform to gracefully degrade and recover from this kind of subsystem interruption. Once state reconciliation was completed and all services were stable, markets were restored by first enabling ActiveTrader and API connectivity in limit-only mode, then re-enabling Mobile and Retail Web. Since this incident occurred at a time when Gemini was transitioning order flow to an upgraded version of the exchange matching engine, the process of state reconciliation prior to restarting production services required reconciling state across two trading systems. After we restarted the impacted systems, it was determined that the messaging system errors led to state divergence of some downstream systems due to how our systems interacted with it. The messaging system automatically restarted, but many internal message consumers and producers required manual intervention. ![]() All three nodes that make up this messaging platform failed at the same time with the same exception. Normally, this messaging infrastructure allows for strong reliability guarantees for applications that depend on it. It is multi-node and generally fault tolerant. This messaging system is responsible for fast, high-capacity, reliable delivery of messages within our distributed exchange platform. Prior to the planned final migration of BTC/USD and ETH/USD to the new platform on Friday, we experienced a failure of the messaging infrastructure that this system depends on. This post provides additional information about the recent service disruption and how we are improving the reliability and performance of our exchange through a series of upgrades. This involved migrating 79 trading pairs on a coordinated schedule to a new platform while continuing to operate our exchange 24 hours a day, seven days a week. In addition to maintaining our trading and custody platform, we have been working in parallel to upgrade our backend exchange platform to improve system capacity and scalability. During that downtime, all our customers' funds remained secure. Gemini has been embroiled in an intense spat with Silbert's Genesis Trading, a crypto lending firm that generated rich returns for Gemini clients through Gemini's high-yield lending product, which is known as Gemini Earn.On Friday, December 10, we experienced a service disruption that resulted in an exchange outage for ten hours. The exchange also faces a legal fight with the Securities and Exchange Commission over an alleged unregistered offering and sale of securities in connection with its partnership with Barry Silbert's bankrupt company, Genesis. Gemini has endured a battle over customer funds in recent weeks. "It was our hope to avoid further reductions after this summer, however, persistent negative macroeconomic conditions and unprecedented fraud perpetuated by bad actors in our industry have left us with no other choice but to revise our outlook and further reduce headcount," wrote Cameron Winklevoss in an internal message obtained by The Information. In early January, Coinbase slashed 20% of its workforce in a second major round of job cuts in an effort to preserve cash during the crypto market downturn. ![]() 11, the day that Sam Bankman-Fried's crypto exchange FTX filed for bankruptcy. Other crypto firms like, Coinbase, Kraken, and Genesis have eliminated positions since Nov. TechCrunch reported that Gemini had previously trimmed its headcount by 7% in July 2022, following a 10% staff a month earlier. Gemini had 1,000 employees as of November 2022, according to PitchBook data, suggesting around 100 people lost their positions. Personal Loans for 670 Credit Score or Lower Personal Loans for 580 Credit Score or Lower Best Debt Consolidation Loans for Bad Credit
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |