SSD Resets During Large Indexing and Exports

[ifinch001]

In the event that someone has encountered this when transitioning from HDD drives to SSDs for Intella purposes. Last week, I switched out 2 HDDs for SSDs, making the current workstation setup 1x PCIe NVMe C:\, 2x Crucial SSDs for cases and exports, and 1x WD HDD for large, long-term storage.

I am finding that when I index on or export to one of these new SSDs, the destination SSD disconnects (e.g. the SDD that contains the index or the SSD that is being exported to). This happens on multiple machines and multiple different Intella cases (using 2.4.2 and 2.5). Event logs show an event 129 in the System log "Reset to device, \Device\RaidPort0, was issued." At the end of the day, this is the result of the drive disconnecting while data is being written to it. All other drives remain connected to the workstation without issue.

Index/export work as expected when using the NVMe or the HDD. Both SSDs (and even a separate machine with the exact same setup, so 4 total SSDs) are experiencing the same issue. I have run the gambit of troubleshooting for the last week from a general driver/firmware/software/SSD perspective and no improvement.

The reason I am posting here is because this seems to happen exclusively when using Intella. As a test, I exported out a data set that causes the issue to the NVMe drive (which worked perfectly fine, but not a long-term solution), then copied the exported data to both SSDs and the disconnect did not occur. And after using the workstation for other purposes, Intella appears to be the only trigger.

I should also note that some indexes work perfectly fine. It seems if indexes or exports are in the ballpark of 40K items, the disconnect occurs.

So, I am inclined to believe that indexing or exporting from Intella is stressing the SSDs for some reason. Anyone else experience this issue? Or is there some setting I need to change or Intella version I should try?

 

[arjohn]

Hi,

We have been testing with both HDD and SSD for a long time but haven't seen this before, nor do we have such reports from customers. Exporting and (especially) indexing do indeed stress the storage system -- but 40k items is a fairly small data set, our test usually involve millions of items.

I'm inclined to think this is either a hardware, driver, firmware or temperature issue. You mentioned that you are seeing the same issue on multiple machines: do all of these have identical configurations, or have relevant hardware in common? This includes: SSDs, disk controllers and motherboards. Might be worth running a disk stress test using a tool like HD Tune, while keeping an eye on SSD temperatures.

[ShaunC]

I've only got NVMe drives in my 4 machines that I can use Intella on, and certainly none of these exhibit this issue.

Sounds rough - what does Resource Monitor show in terms of disk IO etc? Can you see any bottlenecks/100% utilisation?

Does reducing the number of crawlers help? Less crawlers may mean less disk congestion?

[ifinch001]

On 12/6/2021 at 6:06 AM, arjohn said:

You mentioned that you are seeing the same issue on multiple machines: do all of these have identical configurations, or have relevant hardware in common? This includes: SSDs, disk controllers and motherboards. Might be worth running a disk stress test using a tool like HD Tune, while keeping an eye on SSD temperatures.

There are 2 machines that have the exact same setup and both exhibit the same behavior. Strangely enough, the second machine will export significantly more items (around 100k) before the destination disconnects. As for a stress test, I did go down this road and everything I throw at these SSDs turns up normal.

 

6 hours ago, ShaunC said:

Sounds rough - what does Resource Monitor show in terms of disk IO etc? Can you see any bottlenecks/100% utilisation?

Does reducing the number of crawlers help? Less crawlers may mean less disk congestion?

Thanks, both of these are good ideas and I will try them out this week.