COMPUTERS, THE AGONY AND THE ECSTASY

[PanBiker]

Is there a Linux community help site or dedicated BBS anywhere where you could post the problem? Seems strange, I take it Linux fully supports NTFS? Not being a user I have no idea.

[Stanley]

I thought something was wrong Tiz and nearly mailed you to see if you were OK yesterday. Glad to hear you are fine but sorry you are having a problem. I haven't done any back ups with V18 so don't know if I have a problem.
Whatever it is I hope you crack it soon!

[Tizer]

I'm alive and kicking, Stanley, although surrounded by 2 plumbers and an electrician. I'm sure your backups will work ok, I think it's more to do with new disks. Yes, linux supports Ntfs, Ian. I'm going to buy a Toshiba disk and hope that works ok like my old Toshiba. If not I'll make it a sacrifice and format it for linux ext partitions. If that doesn't work I can put it back to Ntfs and give it to one of the family!

[Stanley]

Some of the backups on my external drives go back to Window Land but they all read perfectly well . Don't ask me what the format is!

[Pluggy]

Linux understands most drive formats. NTFS is taking over from FAT as the lowest common denominator of drive formats. I haven't had any problems with reading anything but I haven't bought new spinning rust in a while.......

[PanBiker]

The main difference between NTFS and FAT is the file cluster handling capabilities, FAT32 struggles with anything over 4GB. Hence the rise of popularity of NTFS as potential file sizes, particularly high resolution graphics increased over the years.

[Tizer]

I've got the Toshiba 1TB external disk now and that's working better than the Seagates, although it's not as straightforward as my old 500MB Toshiba. The old one would accept what you threw at it, just buzz along at a steady rate and eventually get the job done. The new disks all start fast and then slow down- the Seagates would grind to a halt and have to be cancelled, the new Toshiba does slow down somewhat but gets the job done in an acceptable time.

What seems to trouble all the new disks most is folders with a lot of large files. I have a Music folder, with three subfolders containing further subfolders of music tracks ripped from CDs to mp3 files. In total there are about 120 CD folders, each about 150MB in size, so the total for the main Music folder is around 18GB. The old Toshiba chunters through this slowly but surely and steadily. The Seagates quickly give up and the new Toshiba struggles, with alternating periods of activity and no activity (although getting to the end eventually but slowly). Fortunately, I don't often change the Music folder so I don't have to include it in all the backups. But the disk behaviour is puzzling and I wonder if it's something to do with limits on RAM buffering.

There's little about this disk problem on the Ubuntu Forum although I did find a recommendation to reformat the disk from NTFS to ext4. LINK I'm wary of doing this for the reason mentioned earlier - I'd prefer my external back up to be able to be read on a Windows PC in case I was no longer able to do the job myself. I can use the new Toshiba as it stands but I'm always a bit frustrated when I can't find why something behaves in what I see as an odd way!

[Stanley]

Try the extra RAM? I've always put twice as much in as recommended and it has never let me down. That may be the cause Tiz. Not a biggie to try it out.....

[Pluggy]

Have you tried it with a memory stick ? Although at the utility end of the market they can be a lot slower than a hard drive.

How are you determining whether it has activity or not ?. I wouldn't put any faith in Ubuntu's gauge. Watching the hard drive indicator light (if it has one - they have been a victim of cost cutting in some instances) and/or listening to the heads moving around are better indicators in my book.

I can't honestly see increasing the RAM having much impact. If it was tight maybe but 8GB isn't tight on Ubuntu. It works pretty well at 4Gb and as Stanley says, double it to be sure.

I could do a comparison here if you gave some figures for file numbers, capacities and actual time taken. I routinely copy 80GB+ off customers machines and they are always NTFS - It can take time, but I go and watch the telly or something whilst its at it.

I've just ordererd the cheapest 1GB external Seagate HDD from Scan https://www.scan.co.uk/products/1tb-sea ... ered-usb30

I'll see if new spinning rust gives me any grief. My desktop has 16GB of ram (helps with Windows 10 VM) but my laptop is 8GB and slower than Tiz's desktop.

[Tizer]

I haven't tried a memory stick, I'd forgotten how `big' (GBs) they are now. I'll give it a try.

The Ubuntu gauges in Nautilus grind to a halt but I've watched Processes in System Monitor and that show periods of little or no CPU activity alternating with active but at times it stays inactive and I have to cancel the backup.* The weird thing is that both of the Seagates I tried did their first backup from my new PC very fast - 30GB/20,000 files - in 6 minutes but every subsequent backup failed. The new Toshiba did the same job in 35 minutes and completed it, no problems.

On Mrs Tiz's PC (the first you built for me) the old Toshiba external disk took 9 minutes to do 10.5GB/16,830 files. With the new Toshiba it started well, quickly doing a lot of smaller folders in about 10 minutes but when it got to a 4GB folder it slowed dramatically and dragged out the backup to a total of 34 minutes. So the old Toshiba is about 4 times faster than the new!

This is the Seagate I tried twice and returned: LINK

*Cancelling and then clicking Remove Safely didn't stop the drive, which is annoying, and a warning popped up when I shut down the PC saying that the drive was still working.

[Pluggy]

Ahh didn't look at the Maxstor. I already have a 500GB USB3 Maxstor that looks the same case as what they are presently selling but isn't the same model number. I've had it about 3 years. it performs to expectations, about the same as a much older USB2 250GB seagate. Although the drive packed up in it a month or 3 back and I pulled it in bits and put an old 250 GB WD laptop drive in it.

[Pluggy]

Just done 47,000 files at 11.7 GB from an internal HHD to the 500 Maxstor. Said it would take 2 minutes initially which went up to 3 after starting. It took 5 minutes to complete but then took about another 30 seconds before it would let me release it. It did appear to stop once or twice which I've always assumed happened when it comes to a big file when copying little ones. I consider that about par for the course. Ubuntu has always underestimated copy times.

[Stanley]

I'll let you know how I get on in June when I do my half year back up...... I refuse to worry about it!
About a year ago Daniel sent me a hard drive so I could copy my image archive on it for him to browse. He uses Apple and so it was an Apple compatible drive but it worked fine, I half expected a problem.
The outgoing server had a brain fart again this morning. I have just checked and as usual if you leave it it heals up. Annoying though especially if you were trying to run a business.

[Tizer]

Using my old Toshiba external drive (500MB, USB2.0) I could transfer 11.5GB in 10 min on our new PC and 10.5GB in 9 min on the old one.

The first Maxtor (1TB, USB3.0) on its first run on the new PC did 29GB in 15 min. On the old PC it slowed down so much it eventually froze the PC and on subsequent attempts on the new PC it was still going after an hour. It also became inaccessible and was exchanged for a new one. The first run of this second Maxtor on the new PC did the 29GB in 6 min. After that it behaved exactly like the new one and was returned.

I then got a Toshiba (1TB, USB3.0) and this did 29GB on the new PC in 35 min and 11.5GB in 16 min. On the old PC (USB2.0) it did 10.5GB in 34 min. At least the Toshiba is behaving well so far and hasn't been freezing the PC like the Maxtor did.

A USB2.0 stick did 12.8GB in 43 min on the new PC and 10.4GB in 29min on the old one.

I still can't get my head around why the Maxtors could do the first run at such a blistering speed and then go slow or die on subsequent ones. It's as if they blew everything on the first run! Also, I can cope with the speeds of the new Toshiba but it seems slower than the old Toshiba, and slower than your new disk. Perhaps it all depends on which way the wind's blowing on the day!

[Big Kev]

Do you have the option to 'optimise for performance' for usb in Ubuntu. In Windows land you can configure usb devices for easy removal or performance...

[Tizer]

Not that I know of, but I'll have a look on the web. Thanks Kev.

[Pluggy]

The 1GB Seagate arrived from Scan and I'm experimenting. Same folder and files as yesterday (~47000 files 10.7 GB). !st copy 3 minutes + 20 seconds release time. Then I renamed the source folder and did a second copy onto the same dive without removing the fisrt copy. Second copy took 6 minutes + 6 minutes to release. Repeated the procedure again, bleargh been copying 18 minutes and its about 20% done, its stalled it seems by the computer, the drive light is still blinking and its making convincing writing noises. Not quite stalled, it is still counting up files but very slowly. Its counting up slowly and saying its got 48 minutes to run after 24 minutes in. Hmmmmm.............

[Pluggy]

The plot thickens, abandoned the copy, cold cycled the computer and the drive and then tried another copy. Still running like a comatose slug........

The drive is showing the expecting 2/10th of bugger all of space used.......

I'm going to try this on an old laptop I gave to my grandaughter which has 16.04 on it.

[Pluggy]

I've screwed the pooch with this drive. It rapidly gets bogged down connected to any computer, Linux or Windows. Scrubbing the partitions and starting again makes no odds. I've even recreated the partitions on a Windoze Box.

It starts quickly and slows down what ever you do now. My old Maxtor drive and the really old repaired seagate continue to work as normal.

Its a conspiracy.......

[Pluggy]

Refusing to be beaten by a damned hard drive, I've perused the web extensively and the conclusion I've reached is :

New Seagate drives use a technology called SMR (shingled magnetic recording) its about cost reduction by squeezing more data onto less spinning rust (no surprise there). It works well and gives good data retention, but when you're writing lots of files using standard disk formats, it slows down progressively. Its nothing to do with the operating system. It would do the same thing on Windows. Using the same 'screwed' drive I can transfer 9 files totalling 40.7 GB in under 6 minutes. What makes it slow is that once it has written an area of the disk once it needs to rewrite the file table for each file and overwriting existing data is messy on SMR drives because of the way it works. Its quick at the beginning because the disk has a cache which speeds things up, once its filled it it then has to rewrite the file table for each file and it crawls. SMR is really a WORM technology (write once read many) and its fast and reliable used like that, but the cost benefits have pushed the envelope somewhat and under most scenarios the drive and computer caching saves it. Unfortunately backing up loads of files repeatably shows them in a very poor light. Its only presently used on External hard drives. SATA drives are PMR (perpendicular magnetic recording) because the shortcomings of SMR would be far more noticeable on a computers main drive. Standard windows type backups which usually compress everything into a very few large files would work well on SMR, Mass copying (which has always been my favoured backup method because of its simplicity and ease of retrieving files) sucks on SMR. Using disk formats especially developed for SMR reduces its problems, but all the current maintream formats aren't.

Seagate are the main users of SMR disks and as Maxstor are Seagate........

For the out and out geeks this explains the various flavours of harddrives : https://blag.nullteilerfrei.de/2018/05/ ... -a-hdd-mr/

[Stanley]

Is one answer going to be using solid state external drives if we get a new one?
I note that the Flight Data Recorders they use now on airliners are SS.....

[Pluggy]

I dunno Stanley, the manufacturers don't let on what technology a drive uses. Stuff everything into a big zip file and put that on the backup......

SSD are something of a WORM device as well, writing is much slower than reading and writing to them wears them out. You can't win when everything is a race to the bottom.......

[Wendyf]

You can't win when everything is a race to the bottom.......

Which, in a nutshell, is why Colin couldn't wait to finish his career as a an electronics design engineer.

[chinatyke]

Wonder which type they use on Boeing 737 Max 8 planes?

[Pluggy]

SSD would be the way to go for any flight data recorder. They survive falling out of the sky at 600 MPH far better than spinning rust.

Thinking about SMR, the caching would make them work pretty well on a main drive, OS's in general don't do a lot of writing. Its all about speed of reading which SMR are good at. Its just dumb mass file copies they suck at.

I'm glad I'm at the tail end of my computer career. When a cheap computer cost £1000 (when fish and chips cost £2) you could make a good living repairing them. When a cheap computer costs £200 (when fish and chips is £6) its as cheap to chuck 'em out and buy new. I had a recent model laptop in bits yesterday replacing a knackered HDD with an SSD. It was a masterpiece of economy engineering. The palm rest, keyboard, trackpad and power switch were all one component, plastic welded together. Simples to assemble but very costly if you just need to change a keyboard when your teenager knocks a glass of coke over into it. So much for reducing CO2 production.