Files, Servers, and the Cloud

Files Everywhere.

Files everywhere describes both a problem and a goal. As homes contain more computing devices—such as a desktop, a laptop, a tablet—, user files can be scattered across locations; worse, the same file can exist in multiple versions making for a management nightmare. This is the problem: files everywhere.

At the same time, any user juggling multiple devices has come across the situation where a necessary file is on the desktop, while what is available is only the laptop. Both of these situations can be fixed by addressing the goal, files everywhere. Through any of various means, home users can take advantage of networked filesystems—it's currently en vogue to refer to cloud-based filesystems—to provide a unified and universal view of your files from multiple devices.

The old normal

In the old-old days, computers didn't have permanent storage like they do today. Data was fed in by hand (or tape, or punch-chards) and printed out in hard-copy. With the rise of mainframes, users often had little need to access files located on a different machine, they would connect to one mainframe and do all processing there. Computers were expensive, and only the largest corporations or schools would have multiple, and they often were used by different groups. If the needed arouse, however, users could copy files manually via tape or in some cases send the file via serial links.

As the Personal Computer revolution gathered stream in the 1980s, this was paralleled in the home. From the 1980s through the mid-1990s, homes often had only one computer, and when a user wanted to transfer a file between the home computer and an office computer, this was often done by first copying the file to a floppy diskette. That's not to say that there weren't technologically better options available, in fact network-transparent filesystems (ones allowing access to files on other machines to users in the same manner as local files) date back at least to the 1960s; the costs of such systems often were beyond most home installations.

As the 80s rolled into the 90s, PCs had found their way into many companies, replacing dumb terminals connecting to a single mainframe. Users needed to find a way of sharing files with the rest of their team, and in some cases, work around the limitations of the native filesystems (such as DOS' FAT). Two revolutionary creations in middle of the decade had set what would dominate two distinctly different worlds for some time to come. In one world, Sun Microsystems created the simply named Network File System; while in another, Novell released its Netware operating system.

The first of these became the standard in UNIX or UNIX-like networks, and support for NFS is still common among modern operating systems. Networks build upon Netware have tended to evolve into MicroSoft-centric networks around various iterations of their Windows operating system, and file sharing in these networks use the Common Internet FIle System (CIFS, formerly SMB for its earlier name, Server Message Block).

What's Normal Now

Home fileservers have become common any various forms, the most common usage being for storing media which multiple people may want to access from many places, for instance: a shared music collection. More ambitious users may use these fileservers much the same way as at work, and may be using them to successfully manage working on documents from multiple locations. As home networking has gotten easier with faster wireless technologies, companies have begun marketing turn-key network attached storage (NAS) servers for home-users. These machines often implement a number of redundant technologies in order to give users the widest variety of options by which to use them, but the two most common means of connecting to them are still NFS or CIFS/SMB.

The other popular option for global file access is the umbrella marketed as cloud storage. These systems tend to be marketed and better suited as back-up systems rather than day-to-day file access. For the sake of this article, the important distinguishing factor for a cloud storage system is that it stores your files on someone else's network, outside of your control and remotely access across the Internet. Access to cloud storage systems are slower than local systems and creates a potential failure point for which you lack any control or recourse in case of failure. Despite these drawbacks, cloud systems have the benefit of being accessible from anywhere, so while they might be slower to access than local network-attached-storage systems at home, if travelling, they might actually be faster than relying on your home network's uplink to access your files.

What you can do

Quite Normal is looking for ways to be as self-reliant as possible, and as such would suggest running your own network storage system, relegating proprietary commercial cloud systems to a back-up role.

Thinking about your Cloud

There are a few aspects you will need to decide before creating your own cloud-like platform. You'll need to determine how you will store your files, how you will access them, and how to determine who can access them. Whether you want to be able to access your files from many devices, or from anywhere physically, also comes into play as this is a security versus convenience trade-off.

There are many ways to manage generic user authentication—LDAP, Kerberos, RADIUS—and this will be a later topic; so let us talk about the other topics now.

Backing your cloud

What will you use to store files in your cloud? If you are looking for on-line access to the files, and all on the same network (or connecting via a virtual private network), NFS or CIFS (Samba) is a reasonable choice, as there is wide availability and an off-the-shelf NAS solution could suffice for a home set-up. In order to access your fileserver, you'll need to also create a VPN solution, which may be one additional layer of complexity in your networking set-up, but this would be a fairly standard configuration and the most widely supported.

If you looking for more transparent file access without worrying about which network you reside, AFS is an option. AFS is usually more difficult to set-up, but connecting to an AFS server is possible from Linux, BSD, OSX, and MicroSoft Windows-based machines with the OpenAFS client software. AFS integrates with Kerberos for user authentication, and both of these systems are designed to be used regardless of from where the client computer is connecting. Another option, also somewhat difficult to configure on the server-side but widely supported without additional software installations, is WebDAV.

These solutions all require a good network connection. NFSv4 and later and AFS will provide some ability to work on slow connections by caching data locally, but when the computer has no connection at all, these systems will not fare well. One solution is to copy files to local disk ahead of time, and copy them back to the network system when possible. This could be automated using rsync or various tools built upon it. Coda is an uncommon filesystem designed to handle these situations (although obviously, it too needs network to synchronize data or cache files it hasn't yet cached).

Accessing your cloud

AFS or WebDAV are usually accessed the same whether on the local network or over the Internet. For AFS, a user authenticates to the server if necessary (via Kerberos), and then accesses files. This process is similar for the user in WebDAV, although the technical details are a bit different. Because client support for WebDAV is broad, from OSX an MicroSoft Windows to Symbian S60 phones, many commercial file storage products (including Apple's MobileMe/.Mac/iDisk) use WebDAV. AFS and WebDAV both have the ability to encrypt the communication between client and server to help protect the confidentiality of your files while accessing them remotely.

While there is nothing that prevents other, traditionally LAN, file sharing systems such as CIFS or NFS from operating over the Internet, most users will prevent access to these fileservers from the Internet directly. The short reason is for concern regarding weak authentication mechanisms, although NFSv4 offers the use of Kerberos as a way of securing both authentication credentials and file data making. If you do decide to block access to your fileserver from the larger Internet, you can provide a Virtual Private Network connection for yourself or your users to use while remote, allowing the remote device access to your network as if it were local.

What will be Normal

Personal computing has wavered between centralized storage (mainframes and the cloud) and local storage (personal computers and other mobile devices). The reality of the situation is that a combination would be optimal for most users. For anybody who travels outside of a metropolitan area, poor network availability and high roaming costs are prohibitive, requiring the use of local drives. Nevertheless, as a primary back-up or file-sharing service, cloud services offer a convenience factor whether personally controlled or managed by another party.

Return Home