Toyota servers run out of storage

author: FutureCar Staff    

Toyota has announced that a server running out of disk space was the cause of an outage that resulted in the shutdown of 14 manufacturing plants across Japan last week. The company stated that the system malfunction occurred due to the unavailability of multiple servers that process parts orders.

The servers went down after regular maintenance work was performed on August 27. However, it was discovered that the maintenance procedure had resulted in the deletion and organization of accumulated data in the database, causing an error due to insufficient disk space. As a result, the system stopped functioning properly.

Since the servers were running on the same system, the backup function also failed, leading to a complete system failure. It is believed that Toyota's production and backup servers are identical, which is why the failover process was unsuccessful.

To resolve the issue, the data was transferred to a server with a larger capacity on August 29, restoring the system. However, this restoration process resulted in a loss of approximately 36 hours of production time.

Toyota has acknowledged the seriousness of the situation, especially considering their just-in-time manufacturing approach. The company's technicians have identified the root cause of the problem and have developed countermeasures to prevent its recurrence. They have also committed to reviewing their maintenance work procedures and strengthening their efforts to prevent similar incidents in the future.

One positive outcome from this incident is that Toyota's initial assertion that it was not a victim of a cyber attack has been proven correct.

FutureCar Staff