Flashing Fujitsu’s PX05SRB series “time-bombed” firmware SAS SSD
Last month I bought a used PX05SRQ192 with 61% disk health from a server recycler. It was very cheap, for $50 each. As the disk arrived, I inspected and saw it has Fujitsu part number in it. I checked the drive and it’s working great. Though I was wondering, why is the disk health getting very low, even though the drive write is not even 10tb?
I researched the drive and it’s part of Toshiba PX05SRB series. I looked up the card series and found this forum post from ServeTheHome https://forums.servethehome.com/index.php?threads/toshiba-px05srb-ssd-failures.43665/ . Now I know why this drive health is so low. The problem in this drive series is it has a bugged firmware, or a “time-bombed” firmware, as they say.
The IBM support page linked https://www.ibm.com/support/pages/required-drive-firmware-upgrade-192-tb-and-384-tb-tier-1-flash-drives-6402-firmware in STH suggested that “As data is written to the array, wear-levelling and rebalancing occurs when a cell has an excessive program/erase cycle count”. So it’s some kind of write-amplification bug eating the drive health.
As I mentioned before, this drive came from a Fujitsu-related. I dug deeper with my best friend, google, and found out that this particular drive came from a Fujitsu Eternus DX storage array series. I search on Fujitsu’s website. I selected a random DX storage series and a popup immediately showed up.
Boom. Same message as the IBM support page earlier. I checked on the document and found my disk.
PX05SRQ192, right there. My particular drive comes with 6807 firmware, so it is affected by the bug. I downloaded the latest firmware from their site to flash on the drive.
This way I was able to get the latest firmware.
Next, I tried flashing the firmware. I’m using 9300–8i HBA card, so I tried to use storcli64 to update the firmware. It says “Firmware Download Successful”, but the drive firmware is still the same. I also tried using openSeaChest from seagate, spent a week researching with no luck.
The next day I tried flashing the drive using sg-utils. However, it failed to write the firmware to the drive.
sending single write buffer, mode=0x5, mpsec=0, id=0, offset=0, len=1245696
Write buffer cdb: [3b 05 00 00 00 00 13 02 00 00]
Write buffer parameter list (first 256 bytes):
43 6f 70 79 72 69 67 68 74 20 28 43 29 20 46 55
4a 49 54 53 55 20 4c 49 4d 49 54 45 44 00 00 00
44 69 73 6b 41 72 63 00 54 4f 53 48 49 42 41 00
01 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00
50 58 30 35 53 52 51 31 39 32 20 20 20 20 20 20
36 38 30 41 00 00 00 00 00 00 00 00 00 00 00 00
21 20 08 20 10 49 37 00 00 00 00 00 00 00 00 00
00 02 13 00 00 0a 00 00 01 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
13 00 00 00 36 38 30 41 00 00 00 00 21 20 08 20
10 49 37 00 00 0a 00 00 00 f6 12 00 01 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
check_file_type: file descriptor is sg device
duration=33 ms
Write buffer:
Fixed format, current; Sense key: Illegal Request
Additional sense: Invalid field in parameter list
Raw sense data (in hex), sb_len=48, embedded_len=48
70 00 05 00 00 00 00 28 00 00 00 00 26 00 00 00
00 00 00 3b 68 07 01 20 00 00 00 00 00 00 78 00
00 1d 32 24 00 00 00 00 00 f0 54 08 00 00 00 00
Write buffer failed: Illegal request, type: sense key, apart from Invalid opcode
Sense key: Illegal Request. At least we’re going somewhere now, it couldn’t flash the drive. As I’ve spent too much time on a single disk, I moved on thinking maybe this is the end of it.
—
Fast forward around 3 weeks. I was able to get more Kioxia PM5-V from Dell for backups. I tried flashing one of my PX05SRQ192 again using sg-utils, same issues. I then googled more and found another PX05SRB-series firmware from Dell from here https://www.dell.com/support/home/en-id/drivers/driversdetails?driverid=6ckwm. It’s for PX05SVQ (V means mixed-use drive). I downloaded the Red Hat flasher and extracted the .bin file to check on what tools this drive use.
I then inspected both firmware on hex editor to find difference, and I found it.
There is extra header with text “Copyright © FUJITSU LIMITED DIskArc. Toshiba”. Other than that, I noticed that they has the same value from first hex until hex address 000000C0. I inspected the firmware further and found some more weirdness from Fujitsu’s firmware.
I noticed the Fujitsu’s firmware file starts on 00000A00 address, while Dell firmware starts from 00000400. It’s very weird because they are basically (more or less) similar drive. I took notes and screened both firmware and found something more on the end of file
Fujitsu’s firmware is mostly empty at the end and there were some random hex at the end of it. The file also ends on 00130200 rather than 0013300 from Dell.
It took a while from my analysis but I think the header and some of random hex on the header are causing the drive flashing to fail and it was designed to be flashed using Fujitsu’s own flashing tool which is not avaiable publicly.
I took action by removing unnecessary hex by 0-ing it out. There were also some hex that are not 0 between the firmware version and firmware type and firmware compile date.
And I also deleted the 0 space and moved up the firmware from 00000A40 to 00000530. It saved some unnecessary empty space and almost the same as Dell’s firmware which starts at 00000400. I exported the firmware and named it test.bin, then I tried to flash it
My disk is on /dev/sg5, so I proceed to test and see if the disk accept the firmware by flashing using sg_write_buffer -vvvvv -m 5 — in test.bin /dev/sg5
Somehow, it did not spit out Sense Key error! Very good! I then proceed to check if the firmware got update or not by using command smartctl -i /dev/sg5
So the flash was success!
It turns out it’s just the header that making the drive to refuse the firmware. I got relieved because this SSD will not be a paper weight, at least anytime soon.