sábado, fevereiro 23, 2013

Project 10 - Raspberry Pi - Which FS to use with USB drives

Pedro, this is for you...:)



Does not have the ability to address more than 2TB using traditional 512-byte sectors.
It's its simplicity that makes it attractive for the new generation of storage mediums based on flash memory. Usually, flash memory devices (think SD/memory cards and USB keys) don't have the fastest interfaces to the computer, and any overhead a filesystem introduces simply slows it down. And, because flash storage devices don't often approach the volume of spinning-disk drives, you don't need advanced filesystems to handle them.
The inability to handle single files larger than 4GB. As a filesystem it's also not the most efficient, and can slow down dramatically when large volumes of files are stored in a directory.
FAT32 is the de-facto for flash storage, providing the ability to address up to 2TB using traditional 512-byte sectors, while maintaining the simplicity and speed of FAT. Though, of course, it also lacks features that modern filesystems have such as security and journaling - but these aren't often needed for flash storage.

exFAT should be a true competitor to NTFS on systems with limited processing power and memory. NTFS on flash memory has been known to be inefficient for quite some time. exFAT’s smaller footprint/overhead makes it ideal for this purpose. Of course, only if your definition of “ideal” allows software to be proprietary and not open source.
You can use ext3 partitions under windows with some special drivers/tools.
Ext3 or third extended file system is a file system used by the linux kernel. This is a default file system in linux for many applications and popular linux distributions.
The only drawbacks to exFAT are that Microsoft has not released it into the public, requiring that companies licence it for use on their devices. This is likely more aimed at digital video recorder type devices, home users get a licence to use it with Windows.
Because the inability to handle files larger than 4Gb, Microsoft introduced exFAT in Windows Vista and Windows 7. exFAT is built on the simplicity of FAT but designed specifically for large volume flash media, the type of which we now find in memory cards for cameras, portable USB drives and so on. exFAT is faster than FAT32 and does a better job of maintaining speed with large volumes of files. In other words, if you're carrying around and regularly use USB keys or memory cards, you should be probably be using exFAT to make the most of them. exFAT basically takes the FAT file system to the next level, adding a large amount of long awaited features that the FAT32 system was sorely lacking. One of the key features for people doing video editing is the support for >4Gb files and much larger partition sizes than FAT32 typically supported, making it much easier to work with modern multi-terabyte drives.

exFAT has been around for awhile, but I've never really heard about it. Essentially, it's a file system that's both readable and writable on any modern Mac or Windows machine (sorry, Leopard users). All you need to do is format the drive on a Windows machine and you're good to go. Note that you can't format it on OS X, you have to format it on Windows for it to be compatible with both platforms To do so, just:
1.    Open up Windows Explorer and right-click on your drive in the sidebar. Choose "Format".
2.    In the "File System" dropdown, choose exFAT instead of NTFS.
3.    Click Start and close this window when finished.
From then on, that drive should work fantastically between Mac and Windows machines. It won't work with Linux unless you install Linux's exFAT drivers, but for most people, exFAT is just about perfect.
You need to select a cluster size. The data is not read at the byte level, but rather in blocks. Generally, the smallest unit of storage is a sector (traditionally 512 bytes, 4096 bytes for Advanced Format drives), but even that is not how files are stored in file-systems, but rather sectors are grouped together into clusters.
Raspberry Pi itself doesn’t really have a restriction on cluster size, but the operating system you use may. Usually, any standard size will work, but typically, it depends on what kind of files you will be storing on it.
If you are going to be storing lots of tiny files, then select the smallest cluster size possible (one sector; 512 for memory cards). This way you avoid wasting space by having lots of partially unused clusters. If you are going to be storing lots of giant files, then use the largest clusters size possible (usually 64KB; I’ve never seen 1MB clusters :-o). That way you minimize the file-system’s file-management overhead which frees up a little more space for storage.
When not sure, just use 4KB clusters; that works for most general-purpose usage which is why it is usually the default.

Let«s format the USB drive with exFAT on Windows 7:

Install the exFat-Fuse FS onto the Pi:

Tail -f /var/log/messages before inserting the exFAT-USB-Disk:

Insert the exFAT-USB-Disk and issue the same line command again:

USB detected!!!

Let's confirm with blkid:

/dev/sda1: ....TYPE="exfat" on the picture above. Confirmed!

Let's create the directory on which I'm going to mount the extFAT-USB-Disk, using sudo mkdir /media/usbazul:

 After that let's mount the usb:

Let's chmod the directory with 777 because it was created with sudo:

Let's confirm the creation of the mount point:

Let's confirm it with blkid as well:

Now I want to load something onto the mount point via sftp. I'm going to upload a video (M4O.mp4 ...) and the usual Andreas Scholl folder:

Let's play one of the files (my wife singing at the piano):

After that I wanted to unmout the usb drive. For that I issued the command sudo umount /media/usbazul, but the mount point was still busy. I had to kill it. In order to do that I had to find out what was the mount point process: ps -ef |grep exfat |more:

then I issued the command sudo kill 3260, which was the mount point process I wanted to find out.

I can take the usb out of the hub and confirm that I can still have the files on Windows 7 that I had on the mount point:

There they are: M4O.mp4 and the Andreas Scholl folder!
Forget the NTFS FS. Waste of time...

Happy hacking. You don’t learn to hack – you hack to learn…


7 comentários:

Anónimo disse...


Funcionou na perfeição. Porque é que o exfat não é mais conhecido? Nunca tinha ouvido falar. Toda a gente usa o ntfs.

o macpup é uma maravilha. como chegaste a ele? Dá para instalar no disco?


Anónimo disse...

Tive um problema a instalar o macpup num pc que tinha aqui por casa e que é mesmo muito antigo. Dá a mensagem "kernel requires features not present pae". Procurei na net, mas não entendo.


Manuel Antão disse...

o exfat só foi introduzido com o win7. Só há pouco tempo começou a ser mais divulgado.

o macpup é uma derivação do Puppy Linux que já uso há muito tempo. Tem um interface gráfica muito apelativa (pretende emular o mac, daí o nome...lol). E sim podes instalar no disco, embora eu não use. Prefiro manter as minhas opções abertas, usando boot por usb. Tenho várias pens usb com vários SO's.

extensões PAE: o teu cpu de 32 bits não permite endereçar memória acima dos 4Gb... A máquina deve mesmo ser muito antiga. O macpup não tem um kernel não-pae, o que significa que estás entalado. Tens de usar outro SO. Aconselho o puppy, na versão Retro Precise, que tem um kernel não-pae.

Também gosto muito do DSL. Mas não sei se o kernel suporta cpu's não-pae. Tens de testar.


Anónimo disse...

qual a linguagem mais adequada para programar o GPIO? Tenho de começar já, senão depois não consigo fazer nada!


Manuel Antão disse...

olá Pedro. Desculpa a resposta só agora.

Para mim o zen das linguagens de programação é o Python. Se queres aprender de raiz, é a melhor linguagem para isso. C, C++, C#, Java, etc, não são adequados para 1ª linguagem.

Se necessitares de ajuda apita. Existe muita coisa na net que podes usar. o tempo de aprendizagem depende do tempo de investimento que quiseres fazer.

Em paralelo existem livrarias específicas para a Pi escritas em Python para aceder ao interface GPIO, para além do tradicional C.

Conheces alguma linguagem ou vais arrancar com uma de raiz? Na 2ª hipotese nitidamente Python...

Ainda te transformas num Pythonista :)


Anónimo disse...

Tens algum livro que recomendes para o Python, tipo Beginner's Guide?


Manuel Antão disse...

Existem vários.

Tens muita coisa na net. Se quiseres um ebook posso mandar-te por email.