25 August 2013

Replacing Big SaaS - How to cut the Google, Apple, Dropbox, Microsoft, ... cords

With a Prism and Snowden inspired kick in the backside I finally got around to establishing some autonomy from the Big Boys with respect to email, contacts, calendar, network storage/sync and other common personal use SaaSs.  No rocket science here, just a consolidation of lots of "which one is best for me" research, "follow the tutorial" efforts and Google and log file problems/solutions to explain how to install, configure and maintain the types of services you get "for free" from Google, Apple, Dropbox and the rest.

This article is an overview of how to accomplish replacing the important Big SaaS, it is not a detailed step-by-step with every command listed.  I reference a number of other web pages and tutorials to help with the harder parts.


Here is a basic overview of the substitutions:

Hosting and OSGoogle, Apple, Microsoft, Yahoo, ...Digital Ocean "Droplets"
EmailGoogle, Apple, Microsoft, Yahoo, ...postfix, dovecot
ContactsGoogle, Appledavical
CalendarGoogle, Appledavical
Network storage and syncDropbox, Copy, Google DriveownCloud

The aspirational criteria I had for the substitutions were:
  • Open source
  • Supported with apt-get or similar installer with an up-to-date stable version available
  • At least some recent community activity and support
  • Positive reviews, particularly as versus their popular commercial alternatives
  • Free or close to it
  • Targeted solutions, not one package that is providing many services (e.g., MS Exchange vs Postfix)
It's also important to keep in mind that these solutions generally won't be as good as their popular commercial alternatives where armies of developers and systems administrators support them and taking advantage of big economies of scale and underpricing.  To take this path you're going to forfeit convenience, better usability, rock solid systems and uptime, macro level security, and "free" pricing for greater privacy and control.

Lastly, there are many more areas that could be substituted and I've not done or written these up yet - I note at least some of them at the bottom of article.

What's Required From You

You have to be able to do the following to get this working:
  • Basic Unix shell commands and configuration file editing
  • Willingness to read various tutorials and how-tos and be able to google for the rest
  • Willingness to pay $5 per month for hosting and another $1 per month for backups
  • Accept having a total data footprint of 15GB or less (or be willing to pay for more storage)
  • A basic understanding of SSL certificates is useful

1. Create an SSH key

Follow Digital Ocean's tutorial to create your own key.

2. Have a domain name ready to use

There are many companies that offer domain registration.

3. Hosting

Set up an account with Digital Ocean (digitalocean.com).  Their basic IaaS virtual server ("Droplet") is cheap, plenty performant for our uses here and their management and provisioning interface is pleasantly usable.

Buy the cheapest cheapest droplet at $5 per month (1 CPU, 512MB RAM, 20GB Disk, 1TB transfer).  This will provide plenty of horsepower and space for the average user.

You might select "Amsterdam" as your region if you thought that might provide a safer environment for your data as opposed to hosting that is based in the USA (Digital Ocean's other sites are in New York and San Francisco).

Select OS "Ubuntu 12.04 x64".  You could probably safely use the newer versions, I've just not moved up to them yet.

Install the SSH certificate you created in step 1.

Enable "VirtIO" if you want.  Whatever it is.

After your new virtual server is created, activate automatic backups for it.  They may only be taken about once per week but they're a bargain at $1 per month.

Set up your new domain name to point to your new droplet IP address.  Digital Ocean's DNS interface is easier than godaddy's.  Configure your domain to use Digital Ocean's DNS.

NOTE: The only thing I don't like about Digital Ocean for hosting is there is no apparent way to cost effectively scale just disk size.  I'd like to keep the memory and CPU of the smallest instance but then easily scale up disk space.  Replacing network storage and big IMAP email archives will exceed the 20GB limit for "power" users.  There are plenty of other providers and some allow a low-performance-high-disk-space specification.  However, among the usual suspects like Amazon and Rackspace along with a number of others I found googling around, I didn't find any in the same price range as Digital Ocean.  Maybe Digital Ocean will add the feature of cost effectively adding disk space only in the future.

4. Basics

Verify you can log in as root using ssh and the ssh certificate you created.

Restrict root login to only allow certificate based logins.

Create a new user that you'll use to do most work from here forward.

Enable new user for sudo use.

Install zsh (or your preferred shell if its not already present) and make it your default shell.  Update your login shell preferences.

Create/deploy another ssh certificate for the new user you've created.

Install ntp.

Install iptables as your firewall.  Digital Ocean has a good tutorial

5. Supporting applications

Before we get to the applications we want, we have to install their supporting applications.

Install postgres - used by davical

Install MySQL - used by ownCloud

Install Apache and PHP - used by almost everything

Install phppgadmin - used to administer the Postgres / davical database

Install phpmyadmin - used to administer the MySQL / ownCloud database

6. Create a free SSL Certificate and install it

The certificate will be used by a number of services we install.

Use this tutorial at arstechnica to create a free Class 1 SSL certificate with startssl.com.

  • startssl.com creates an S/MIME and authentication certificate and automatically installs in your browser.  You might want to save the authentication certificate someplace secure.
  • Certificate only good for one year - just remember you need to renew it each year (all your services dependant on a valid SSL cert will stop working when cert expires)

7. Email

Note: I don't typically use webmail, so I didn't bother installing a webmail service.

Install postfix - see Digital Ocean tutorial

Install dovecot - also see Digital Ocean tutorial, my user comments on dovecot

Update DNS MX record.

Adjust iptables firewall settings - see Digital Ocean tutorial

  • I found "apt-get install mail-stack-delivery" did the heavy lifting for me here.
  • Make sure you un/comment out exactly what you want in /etc/postfix/master.cf
  • Increased value of mail_max_userip_connections from 10 to 30 in /etc/dovecot/conf.d/01-mail-stack-delivery.conf due to an IMAP error limit popping up in OS X mail.
  • Digital Ocean has subsequently created a tutorial for iRedMail - looks easier to set up and includes a webmail interface
Note: not added in spam filtering yet.

8. Contacts and Calendar

Install davical.

I looked at and discounted the following:
  • calendarserver - depends on extended file attributes; apt-get exists but doesn't appear to be maintained
  • radicale - no backoffice, feels too barebones
  • baikai - No apt-get; synology's choice for their sync app
  • ownCloud - ownCloud already looks bloated

9. Network storage and sync

Install ownCloud.

The goal here is secure and pervasively available files.  Like Dropbox and the paid version of BoxCryptor - both of which are closed source and therefore non-starters with my stated criteria.

You can create an encrypted filesystem on your main OS, ideally once that can be used by several OSs and place the system in ownCloud network synced storage.  When choosing an filesystem, it's important that the encrypted filesystem is in separate files or some type of chunks, not one big blob (like truecrypt) as big blobs don't sync well when you have concurrent clients syncing.  Ideally you want a filesystem that encrypts file names, content, and inode structures separately in small efficient pieces.  While interesting, I'm seeing enough limitations and sync problems with OS X's encrypted sparse bundle approach that I don't recommend it (use EncFS if you can; else use BoxCryptor even though its closed source).

iOS and Android Support

The above approach is fully supported by iOS and Android devices using standard protocols:
  • Managing email via Secure IMAP
  • Sending mail via Secure SMTP
  • Calendar via calDav over https
  • Contacts via cardDav over https
  • Network storage and sync via ownCloud iOS/Android apps; runs over over https
This probably goes without saying, but assume you'll lose your device at some point.  Think about what is on the device and how easy it is to access it.  Do you use a PIN with a self-destruct after so many incorrect entries?  Do you have logins and passwords in Contacts or Notes files?

Maintenance Notes

You will have to renew your startssl.com security certificate each year.

Spin up the occasional backup on another droplet to verify backups and the restore process works.

Security Notes

Nothing is 100% secure.  The approach I've presented here has two big problems:
  • Hoards of security specialists at the big companies will collectively know more about security than you or I ever will.  Security exploits of fairly new and not widely used applications like ownCloud and davical are possible.  You're therefore effectively trading off having thousands of staff at the big SaaS providers or the government having access to your data vs relying on common sense security basics to stay safe.  In this case, we've done the basics:
    • We're running the iptables firewall with only the bare minimum of ports open
    • All coms over SSL
  • We're not storing the actual data on the server in an encrypted format.  Ideally we'd use an encrypted filesystem on the server so that the hosting provider couldn't snoop disk data.  Of course, decrypting "on the fly" as applications access the encrypted disk is also a risk, but without using your own secured physical server you are stuck with that problem.
I've not yet installed openvpn.  Could switch access to potentially vulnerable apps like Davical's backoffice, phpmyadmin, phppgadmin to VPN only access.  I did add in .htaccess/.htpasswd files across the backoffices for slightly better security.

Lastly, this is pretty obvious, but use long passwords with lots of variation between passwords and a mix of letters (upper/lower), numbers, and symbols.


Google, Apple, Dropbox and others provide a great no/low cost option for services like email, personal information management and network storage.  Signing up for an account with Google is a lot easier and cheaper than the approach outlined above.  You get most of these services "for free".  So if the thought of Google, Apple, Dropbox and others reading your emails and documents and enabling governments to do likewise doesn't bother you at all, then by all means use their free services.

However, if you think you have a right to personal information privacy without business and governments having the ability to read it then you might want to consider implementation of the approach in this tutorial.

What have I missed and what has worked well for you?

23 August 2013

Nipping at Dropbox's heels

There is a real resurgence of cloud storage taking place since Dropbox first launched some years ago and quickly rose above (and for the most part crushed) its competition at the time.  I'm going to highlight what I thought was going on earlier in the year, what's happened since then, and then highlight two new services.

Earlier this year I wrote about p2p file sharing as a threat to Dropbox, particularly the Cubby service.  The problem is, Cubby got their pricing model wrong (you have to pay for DirectSync, their p2p product) and haven't changed it.  Too bad for them, they'll never compete successfully with Dropbox in this way.  I don't expect them to disappear any time soon, but I do expect them to idle along as a bit player in the cloud storage space.  However, if they switched their Directsync feature into their "Basic" (freemium) package and focused on a pay model of central mirrored/subset storage for backup and faster syncs of critical data, it could still be a different story.  But probably not for much longer.

I also looked at BitTorrent Sync earlier this year.  I had various issues with it and while I like it, and ultimately lost trust in it due to regularly disappearing files from a folder I had under test between several machines.  I no longer use it although in the right circumstances I might still use it (e.g., mirroring a lot of content between multiple locations for a limited duration and easily verifiable results).  I continue to think that BT Sync will remain in the tech fringes for now - they will continue to be hampered by the need for a central server (is the server really secure?!), shared secrets approach (who can see my secrets and my files?!) and lack of open source (what am I really installing on my box?!)

Two interesting things have happened since then in the cloud storage world:
  1. A new service called Copy has been making the rounds.  The good news is that you receive 20GB up front (if you activate your account through a referral link; otherwise you receive 15GB for direct signups).  This is 10x more space than Dropbox's upfront free 2GB of space.  I'm using Copy a lot now rather than Dropbox.  Copy has built their own infrastructure rather than using Amazon's AWS as Dropbox has done, I'm assuming that's given them much better cost structures and allows for this much greater level of freemium marketing. After a few weeks of testing, I switched to using it actively.  I've now been using it for a few months, no issues.  [Disclosure: If you click the links above you get 20GB rather than 15GB because its a referral and yes I get free space for the referral - thank you very much!] [Postnote: Warning! See comment below]
  2. On the p2p storage side I stumbled on ownCloud.  I have it running on my own cloud IaaS server (Digital Ocean - working great and dirt cheap at $5 per month for a basic server instance) and have a few nodes connected to it.  It feels clunkier than Dropbox and Copy, but it does seem to work and most importantly, it's free! I'm testing it now and haven't switched to using it as a primary cloud file store yet.  [Note: I've only used at as network storage, not for Contacts and other features.]
Last thing to note - I've given up on BoxCryptor and moved on to using an OS X Extended/Journaled 256 bit AES, sparse bundle disk format on each of these cloud services.  I've had to give up multi-platform access that came with BoxCryptor but on the positive side I'm not paying a yearly subscription to BoxCryptor and I have filename encryption as well.  I'm currently testing volumes in Dropbox, Copy, Google Drive and ownCloud, so far so good with all of them. [Postnote: read comments for a pain point on this approach].

So that's how things look in August 2013 for best practices in cloud storage:
  • Use Copy.  It works.  Much more storage for free.  Good for non-technical folks.  Or just keep using Dropbox if you have enough storage with them and/or don't mind paying for your storage.  [Postnote: Warning! See comment below]
  • If you're technical enough, use ownCloud on your own server.  It looks viable for a do-it-yourself if that's important to you and it's opensource as well.
  • If you're technical enough, value security, and don't need multi-platform access to your files, use an encrypted multi-file filesystem like OS X's sparse bundle format to store your folders and files [Postnote: see comments].

25 March 2013

Employee Contact, Profile and Directory Information (via Google Apps for Business)

After spending the usual unexpectedly long period of time to figure out the structure of Google Directory, Contact and Profile management for Google Apps for Business, I thought I'd share a summary of how it ties together.  I also provide references to help you figure out synchronising contact info and a few thoughts about the risks of using Google Plus Profiles.

(Please note this blog entry is about using Contacts, Profiles, and Directory in a business context, not for personal use.)

The Basics

If you just want to make Google Apps and Contacts work, read this section.  Basically, you will need to learn about Google Contacts and Google Plus (Profiles/About).

  1. Log in to your Google company account (YourName@yourcompany.com) to get started (e.g., via your Contacts page)
  2. If you're not already there, select Contacts from the top horizontal Google services menu.  You'll note "My Contacts" in the left-side navigation.  Use the "My Contacts" group to manage your own personal and private contact information.  By default you see a list of contacts that you have sent/received email (TBC). Anything you add to default employee contact information is private just to you and is not shared.  Google Contact information can be synchronised with other contact management software/platforms like Apple iOS, Google Android, and Microsoft Outlook.  The level/quality of sync is good and it's consistent across at least OS X, iOS and Android (see below for some helpful links on Contacts sync).
  3. Use Google Plus (select "Profile" in left-side navigation, then select "About" from horizontal menu) to set up your own Google Profile that can be shared with others.  The information in your profile should be work focused (use your own private Google account for personal information).  By default some of the less sensitive information you enter into your Google Plus Profile will be publicly shared - this can be limited through Profile settings.  You should take a look at the public version of your Google Plus Profile to make sure you are comfortable with the level of sharing.  Your Google Plus Profile is not well synchronised with other devices and contact software via standard sync mechanisms.  
  4. Unfortunately, there is no sharing of Google Contact information between people (employees of a company), i.e., a centralised and robust employee directory with employee details.  However, you can use your Google Plus Profile to add and share your contact information with others via Google Plus via web browser access or using Google Plus apps on phones/pads.

Further Details

Google's way of managing employee ("Contact") information is confusing.  Three fundamental collections of contact information (Directory, Contacts, Profile) are distributed across three separate yet partially-integrated access points (Control Panel, Contacts, Plus).  Here is how they tie together:

1. Google Contacts (aka "My Contacts").  Available to all employees.  Contacts are created (I think) in two ways.  First: In the "Directory" list of employees, you change or add any field of information (except Circles).  Second: you send or receive an email to someone on the Directory list (TBC).  By putting someone in the Directory list into a Circle, I think this sends a "join circles" invite to the employee but doesn't create a Contact.  Contacts are used by employees to store their own private information about other Google Apps users in the company or any contact information about anyone (employee or external) the employee wants to add.  Employee created contact information is not seen by or shared with other employees.  The information can be sync'ed with other devices through at least CardDav (Apple iOS), Android, and Outlook.  Sync quality is good.

2. Google Plus Profile (select "Profile" from left navigation and "About" from horizontal navigation).  Employee can self-maintain information about themselves in their Google Plus Profile.  Profiles are connected to Google Directory and Contacts.  Particularly useful is the "Contact Information" section in the Profile to enter phone numbers.  Google Plus Profiles are not fully integrated with Google Apps.  It appears that Google is planning to use Google Plus Profile (rather than Google Contacts) to share contact information between people, including employees.  Unfortunately, profile information doesn't sync into your other non-Google contact management software or devices.  Instead you just get a link to the user's profile.

3. Directory (aka Directory Profile).  Accessed from the left-side navigation of Contacts.  Available to all employees.  Directory is a list of all users associated with the company/domain in the Google Apps.  The Directory shows the same user list managed by the Administrator in the Google Admin Control Panel.  By default a user's Directory form shows name, physical address, email address, Notes and attached user profiles although you can add any type of contact field you want.  If you enter or change any information on this form (other than a Circles addition/change), a personal private contact in the My Contact list is created as a derivative of the Directory entry.  Directory entries also connect company employees to their Google Plus Profile.

4. Google Admin Control panel, User Management.  For Google Apps administrators only. This is where employees are first added to Google Apps by administrators for the business.  The only only useful contact information stored here is the employee's name, primary email address and email address aliases (aka "nicknames").  This function is primarily used to add, delete and otherwise administer employees.  Administrators also use the control panel to enable/disable contact sharing (which is enabled by default) and specify related permissions.  Note that "Contact Sharing" is really misleading - it should actually be called "Directory Sharing" because all it does is expose Directory entries (and related limited information) to your users via the Contacts function.

Using "Search" in Contacts makes this really clear.  You can see when and how employees and their information is split between the three areas: My Contacts (Contacts); Circles (Profile); Domain Contacts (aka Directory - more confused naming!).

Tips on Sync

The following are references to instructions to set up sync on various platforms:

  • iOS/iPhone/iPad - Contacts syncing with CardDav.  Don't use the older MS-Exchange way of syncing.  If you have problems, see also:
    • http://support.google.com/mail/bin/answer.py?hl=en&answer=2753077
    • http://support.apple.com/kb/HT4872?viewlocale=en_US
  • OS X
    • Make sure you're running the latest version of OS X
    •  http://www.tuaw.com/2012/09/28/google-now-supports-carddav-making-it-easier-than-ever-to-import/
  • Android phones and tablets - as the others, works fine for contacts but not for Google Plus Profile information
  • Outlook and Blackberry - not tested, but I assume works it works fine for Contacts and not for Plus profile information


Using Google Plus in the enterprise is somewhat risky for several reasons:

  1. By default much of the profile and other information is publicly shared by default.  While this can be restricted by users, it can't be restricted at a global administrative level except by turning off all Google Plus access for all users in the domain.
  2. It appears that Google is using Plus and Apps 2013 to create an environment where you may be required to upgrade/buy proper management of the two in the future (so-called Google Plus "Premium" features).
  3. Google Plus itself and the integration of Google Plus into Google Apps is fairly new, not formally part of Google Apps for Business and therefore unsupported at the same level as Google Apps.  You can also expect it to change as the problems outlined here are sorted out over time.
  4. Managing contact information using typical tools (Google Contacts, MS Outlook, OS X and iOS Contacts) and Google Plus Profiles is clumsy, fragmented and requires users to learn a new tool (Google Plus).  Also users are being asked to self-maintain their shared contact information instead of having someone else do it for them.  As a result users may not see sufficient benefits to start using Google Plus.
However, without looking at a bolt-on Google Apps extension for contact sharing, there is no other option other than using Google Plus profiles to share contact information between business employees. 

You may also have other reasons to press your users toward Google Plus such as enhanced employee collaboration that will encourage the adoption of Google Plus Profiles.


Google Contacts does not provide a shared/centralised employee information management tool for your business.  CardDav and other types of contacts integration between Google and applications like OS X's Contacts doesn't propagate the Google Plus Profile information.

If your business can tolerate the risks, you can use Google Plus Profiles to have your employees self-manage, centralise and share their details.  You can use conventional browser access, Android, or the Google Plus app on phones/pads to access the resultant shared contact (profile) information.

Until Google has improved employee/contact management, products like MS-Exchange will continue to be the de facto choice for employee information management for larger businesses.

05 February 2013

People performance - key measurement points

I was recently asked how I measure the performance of people in IT.  While there are many related aspects of people management (e.g., setting objectives and measuring progress, remuneration change process, conducting a formal evaluation, career planning, motivation, creating shared values/methods, leadership and non-tech attributes, work prioritisation, KPIs, ...), the following is focused on key areas of performance measurement.

For each role, you want to have 3-6 top level areas of measurement - enough for meaningful evaluations but not so many that evaluations become too time-consuming and lack focus.  For each area of measurement, collaborate with the people in that role to list the area's archetypical traits.  Determining and agreeing the areas and the traits are a collaborative exercise although there will likely be recurring themes.

I've not included the process of setting and measuring against objectives in this article.  Objectives are very contextual with the work at hand and will change frequently.  You can list high level objectives and measures at the start of a review process and/or weave them in as examples in each area (particularly #1).  Objectives are the vehicles that demonstrate effectiveness in the five areas below.

The following are my five areas for technology roles along with some fleshing out I've used in both informal and formal reviews.  I've also added weightings for each area, although these might vary depending on the specific role (e.g., a manager of many people might elevate areas 3 and 4).

1. You deliver (40%)
  • Delivery is the most important item on this list of 5 areas
  • You get things done, over and over again
  • You deliver consistently, not in fits and starts.
  • Your deliveries enable additional revenues and/or secure existing revenues (but ultimately, working on a project that contributes big new revenues to the business wins)
  • You take ownership and show responsibility for what is owned
  • You consistently demonstrate integrity, delivering what you say you're going to deliver; however you can also…
    • be flexible and you're not afraid to change your priorities and commitments to do something that is even more valuable to the business at that moment because you can...
    • communicate effectively with stakeholders to come up with alternatives and/or renegotiate a delivery "win" based on changing circumstances and new information and priorities
  • You know when to ask for help and when to negotiate delegation of your responsibilities to others
  • You're focused on delivery and don't allow allow lower priority tasks to interfere with delivery; conversely, you keep an acceptable level of balance with your other objectives
  • You know why what you're working on is important to the business and can explain it to anyone that asks in non-technical business terms
  • You innovate to deliver new products and solve problems, but don't waste time re-inventing wheels
  • You remove, bypass or fix roadblocks and slow-down points
  • Everyone who pushes themselves and their team will sometimes fail:  When you do:
    • you take time to learn from it so you don't repeat the same mistake
    • you're passionate that you're colleagues learn from your failure as well
  • You understand and demonstrate good judgment and flexibility around commercial and technical trade-offs
  • You are comfortable making decisions to enable delivery progress in an absence of complete information
2. You are the go-to person (20%)
  • You're the expert, the master, the guru for your area(s)
  • People trust you and your work
  • You figure things out and solve problems
  • You are the innovator, the break-through thinker in your area
  • You're approachable and can talk at various levels (deeply tech to simplified non-tech)
  • You solve way more problems than you create
  • You have a passion for learning as much as you can in your area(s), but...
    • you exercise good judgment on spending time learning in areas that will help the business versus areas that really only benefit you
    • you don't expect to delay deliveries by spending time learning areas that aren't even distantly applicable to your responsibilities
  • You maintain a top-of-mind list of priorities for your area, improvements and maintenance
  • You know where your main areas of technical debt exist, proactively flag risks, and can present solutions and support business cases on how to remediate it
  • Your mastery radiates confidence, not arrogance.
3. You play nice with others (15%)
  • You make the whole greater than the sum of its parts
  • You think about how your decisions and implementation will impact others and collaborate on difficult choices
  • You're interaction with others increases their productivity and in turn the overall value of the company
  • You actually like people!
  • You share your knowledge freely; you don't stay silent waiting for someone to ask the right question or hoard your knowledge to increase your value
  • You help others, including proactive identification of problems and helping where you can
  • You have several informal mentoring arrangements helping others out on a regular basis
  • You enjoy recruitment and want a say in who joins your team
  • You like giving and receiving feedback - from informally over a beverage to more formally during 360 degree reviews
  • You recognize that different people have different communication styles and strengths and you freely adjust yours to maximize each interaction
  • You look for opportunities to interact with new faces through internal/external coms/blogs, meet-ups and hackathons
4. You have a great attitude (15%)
  • You're generally positive about your work, your colleagues, the company, it's products, and it's customers
  • The first thought in your head is "yes" when asked if something can be done, followed quickly by trade-off thinking and a view on how it can be done.  It's not "no", followed by "here's why we can't do it".  You think "we can do this".
  • You freely give credit to others and recognise and are transparent about when you're standing on the shoulders of giants versus when you are the giant
  • You're passionate and exited about a majority of your responsibilities, recognising that sometimes you have to roll up your sleeves and do some grunt work.
  • You're "present" in interactions, actively listening and participating
  • You're fair and firm, treating people and situations equally
  • You produce more energy than you consume
  • You keep cool and professional under duress
  • You're ambitious, pushing yourself and challenging those around you in a positive way
  • You're self-motivated and motivate others around you as well
.5. You're curious (10%)
  • You recognise that learning is motivated by curiosity, an innate desire to understand how things work
  • You pick up product and domain specific knowledge, at least enough so you understand the company and their products through the eyes of the customer
  • To sate your curiosity you develop relationships across the business.  You learn about other functional areas in the business and mentor others about technology.   As a result you improve your effectiveness at communication across the business.
  • You understand how things get done across the business, how processes and prioritisation work, who the decision makers are
  • You see how the business makes money and how your contribution generates revenue
  • You think about why your (potential) customers go to the competition rather than your company
  • You're interested in what is going on across the business, not just in your area but cross-functionally
  • By exploring new areas you're curious about, you contribute innovative customer-valued solutions
Ultimately, a company must deliver product to customers, not just be a great place to work with nice people.  As a result, you'll notice that I've weighted delivery and talent (60%) over the softer skills (40%).  When creating a new product/service from scratch (e.g., at a startup) with just one or a few people, you'll probably weigh 1 and 2 more than anything else.  When you start scaling up the team size, 3 and 4 start becoming more important.  And ultimately, to be considered a strategic team, you need people excelling in 2 and particularly 5, because that's where your innovation will come from.

Remember to that not everyone excels in all these areas from day one (or 20+ years for that matter…), but it's a good starting point to encourage your team toward greatness.

04 February 2013

A Closer Look at BitTorrent's SyncApp

A few hours after publishing my previous blog article on p2p File Sharing - a Dropbox Killer?, I was very proactively contacted by Kos Lissounov, in charge of development for BitTorrent Sync.  I received a SyncApp tester invite and was able to test-run the product on three devices in a sync group (2x OS X, 1x Windows 7).  Kos and I also had a good and professional back-and-forth of emails and he provided thoughtful answers and comments.  It wasn't my intent to dive down the rabbit hole with SyncApp and BitTorrent security models, but some hours later...

I'm pleased to say that the "pre-alpha" version of BitTorrent SyncApp worked fine for it's main purpose at this point - quickly move around lots of files and data between a specific group of devices.  In particular, bringing a third device into the share group (all on same LAN and nicely linked together - no need to hit the cloud to download files) during my tests ran even faster with two sources of data available to bring the third device into sync.

However, to clarify the key assumption I had when writing the p2p File Sharing - a Dropbox Killer? article:  Can p2p sync replace my day-to-day use of Dropbox, but with unlimited data and at much less cost or for free?  Implicit in that assumption were all the usual points of comparison in the back of my mind: features (vs Dropbox), Just Works (stable), cheaper (for big data sets under sync), faster, more reliable, better usability, more secure, etc.

Based on this, here are a few additional comments about Bit Torrent SyncApp:
  1. Empty folders are not synced.  Per Kos, this deficit is recognised and will be added.
  2. Version conflicts are simply ignored.  However, rather than warning or in any way indicating a conflict, the client just silently ignores the conflicted file.  Version conflict management is apparently tough to implement.  I don't think any type of merge function is required, but I do think a visual warning and changing file names to highlight the conflict is.  Just imagine the issues trying to unpick file conflicts inside of BoxCryptor's "Package" folder with silent failure and removal of a single file from synchronisation...
  3. Their is no API yet.  One could imagine an app simply accessing SyncApp folders via the filesystem on bigger devices (no API required) and using a secret to access a set of folders/files on a mobile device via an API.
  4. Usability is generally a big deficit.  Features like iconic representation of sync state (as done by Dropbox) in a Finder and File Explorer window aren't present.
  5. A "relay" server is required to connect devices via shared "secrets" if the devices are not on the same network and if the peers haven't otherwise previously communicated with each other.  Of course, if one device is a phone or laptop with regularly changing network parameters (on the move between networks), the relay will have to be used to link up the devices.
Items 1 and 2 above for me are showstoppers if my use case is to replace Dropbox with SyncApp.  Items 3 and 4 are painful, but might be tolerated for awhile.  Item 5's relays and "secrets" is tricky to judge because I don't fully understand the security implications.  Let's drill into relays and secrets a bit more.

Relays are a key enabler of the BitTorrent SyncApp approach.  They perform the following functions:
  1. Recommend (not approve if I understand the protocol correctly) the sync of a folder/file set between devices by seeing identical encrypted secrets and recommending the two devices sync with each other.  The two devices must still directly authenticate each other (without the relay involved) - the specifics of this authentication are unknown to me.
  2. Facilitate/broker communication between devices that can't otherwise discover or communicate with each other that have the same secret - deal with firewall issues just as BitTorrent clients do today.
  3. Relay (SHA256 encrypted) information between two devices if the two devices can't otherwise send information between each each other.
Relays do not see any unencrypted data.  They do see and manage SHA256 encrypted "secrets".  Relays store all their information in memory, nothing is persisted to disk.  If the relay goes down then the secret relationship between devices may have to be rebuilt (depends on the device to device access and what specific functions the relay was required for in a given setup).

The concept of the shared "secret" is interesting.  It is a way to enable devices to join in a group to share content.  It has a similar feel to a Bluetooth PIN that is asserted by one device and entered by the other to allow communications between them.  Each folder (and subordinate folders and files) has a unique secret.  Relays (for now a public/shared one run by BitTorrent) are used to coordinate devices with the same secret that can't otherwise find and/or communicate with each other.

I can see two security holes with the "secret" approach:
  1. A secret could be sniffed from the wire and used by a malicious SyncApp client to attempt to join a group of devices with the same secret.
  2. The relay manager (for now the BitTorrent company) could manually insert malicious devices into the relay's device management system.
Kos clarified that the secret is encoded via SHA256(SHA256(secret) - therefore the password (or "secret") is stretched but not salted.  Also that it would be possible to get a list of peers but in order to join a sync group you would have to decrypt the AES256 handshake with the peer with the key SHA256(secret).  Again, I don't know the details of the protocol that engages between the two devices to actually approve joining a sync group so actual joining may be blocked.

Regardless, I remain uncomfortable with the security provided by the "secret" approach as it is today without fully understanding the protocol and implications.  This is also a showstopper for me with respect to using SyncApp to replace Dropbox at least for sensitive information.

Kos indicated a "Should device X be allowed to join this group?" a challenge will be added to SyncApp in the future to help address security concerns.  People/companies can also run their own relays meaning that they can control everything for their sync groups.  However, I think without a central service (relay or otherwise) to provide authentication and authorisation for users, devices, and secrets the product will remain limited from a usability and security perspective.  The Bit Torrent company's obsession with fully distributed, no-master, p2p approaches may really limit long-term market acceptance due to usability and security limitations.  I believe it will also limit their ability to see product adoption beyond a technical community (in which it may excel, just as it has with BitTorrent itself) and in turn be unable to monitize the product.  Even if BitTorrent did put in a central auth server (and even nominally charge for it to make money), would it be trusted given their brand position?  This is where products and companies like Cubby and Skype may have an advantage.

Even more than before, after this review of BitSync I think services like Dropbox, Box.Net, and Skydrive will struggle to compete with p2p sync as their whole business model is tied up in users consuming and paying for cloud disk space.

One use case for which BitTorrent SyncApp excels today is for a fairly technical user to simply keep a group of media files in sync, for example a photo archive on your laptop while you're on vacation being synced (automatically backed up when you have an Internet connection) to a PC running at home.    None of the above issues hold back adoption of SyncApp today for this use case.  In fact, switching to SyncApp for bulk media and other big files (e.g., install images, video, audio) and using Dropbox just for docs and simple workgroup collaboration is a good possibility for me once I'm comfortable SyncApp is sufficiently stable and Just Works.  Of course, even if SyncApp or another similar p2p product closes the feature gap with Dropbox, I still couldn't eliminate Dropbox completely because their first mover advantage is incredible and they are really bedded into the Way the Internet Works now.

If SyncApp can get past their "must be p2p only with no central auth server and keep track of nothing" view of the world and add in some of the features I've covered in this entry, I think it could become a viable Dropbox killer and be meaningful part of a "post cloud" Internet world.

01 February 2013

p2p File Sharing - a Dropbox Killer?

I've been a big fan of Dropbox since it came out, even with the security ups and downs along the way.    They offer plenty of storage for free and it Just Works.  However, two recent announcements have got me thinking about how a competitor might go after Dropbox and other similar (mostly inferior!) products like Box.Net, Microsoft Skydrive and Google Drive.

Background - Some Recent Announcements

The first announcement was from a company called LogMeIn, and their new network storage and sync product Cubby.  Cubby's subtle twist versus Dropbox is that they offer p2p syncing called "DirectSync" (no "cloud" centralised data store required).  LogMeIn has been around for a long time and their product/service Hamachi has been a semi-tech way to set up a VPN between systems - basically a "Goto My PC" for a slightly more technical audience.  Here is the important bit: they have a long history as a p2p communications enabler (UDP hole punching, STUN, TURN, and related tricks) to make the p2p Hamachi service (and now Cubby) work in a behind NAT and firewalls.

The second announcement was by BitTorrent and their new Sync product.  BitTorrent is of course the eponymous creator of the bit torrent protocol, very successful at creating a broadly adopted way of sharing files.  It is also a p2p, no cloud required, network sync product.  BitTorrent, the underlying protocol, also has a very well established p2p communications enabler.

(Postnote: See also subsequent blog article A Closer Look at BitTorrent's SyncApp for related information.)

Now, who is the biggest, well-known p2p service out there, one with a very effective p2p communications enabler scheme, plenty of money and technical capability behind them?  Skype of course, with Microsoft's deep pockets and Skydrive capability.

Dropbox: You've been warned.

Network storage and sync: Cloud vs p2p

Of course, we don't really need a cloud service to sync directories and files between multiple devices.  rsync and boatloads of scripting around it has been been around for many years.  What Dropbox and others did was combine rsync, a cloud-based backup and control location, and It Just Works software with sufficient and simple usability to make it all work.

Skype has shown us that a p2p approach, with highly functional p2p communications enablement  to hook everything together, works just fine.  Articles on Skype discuss this very point of how they keep their infrastructure costs down by only setting up communications between two people (two devices), not actually handling the communications data itself unless they have to (setting aside group coms, super nodes and must-do coms relays to simplify a bit).  Skype wisely focused on value-added services like Skype Out to generate revenue.

BitTorrent has shown us that a flexible and lightweight p2p connection broker service can copy (sync!) massive amounts of data quickly.  And because p2p "cloud-based" components (mostly) only manage connection and data location coordination and don't manage the actual content like Dropbox does, they are light, inexpensive, and resilient (especially against content-rights litigators!).

What's the Minimum Viable Product feature set to push p2p sync over the line?

In addition to the fundamental idea of p2p sync, I think there are a few additional key features to create this new, hypothetical, revenue-generative p2p-based Dropbox killer:

1. Just Works
  • Dropbox absolutely gets this one right
  • With less reliance on the cloud, this should be even easier in a p2p world
  • Stable (no Microsoft BSODs), fast
  • Doesn't crush system performance by pigging all available disk I/O for indexing, CPU for sync calculations, Internet or local network I/O for sync transfers (if possible, sync via directly on a local LAN with no public Internet bandwidth required to shovel data between devices)
  • At least the efficiency of rsync in the sync approach (e.g., block level not file level syncing)
  • Works in corporate environments with strong firewalling (e.g., relay if necessary, uses typically unblocked outbound ports 80/443)
2. Pervasive - available on all high-use platforms and applications
  • Desktop, Laptop - OS X, Windows, Linux
  • Mobile - OS X, Android
  • Application access: simply local filesystem use where possible (desktop/laptop) or an API for sandboxed environments like mobile
3. Secure
  • All data sent over the network is encrypted, both to peers and to the cloud based service coordinator (e.g., public/private key)
  • Service coordination:
    • Authenticates you as a valid user, your account information, including billing information for billable value-add services
    • Authorises your nodes you want to keep in sync
    • Authenticates and authorises other's nodes you are sharing data with
    • Knows nothing about your data (no relay function unless required - and even then only passes through encrypted information)
  • All data is encrypted at rest in each participating node (see BoxCryptor - a big hole and opportunity in Dropbox's offering)
4. Usability
  • BitTorrent really struggles here (indexing, trackers) but a viable connection coordinator (e..g, Skype, Cubby) would solve this problem.
  • Tight filesystem integration (like Dropbox)
    • Async recognition of folder/file changes
    • Iconic representation of sync status in Finder/Explorer windows
  • Clear, simple communication of sync status between devices within a client and potentially iconically:
    • Which device(s) has the master/newest version of a file (think BitTorrent's Seed/Leech)?
    • How synchronised is is a file, directory, or everything? (as a % complete)
    • How "safe" is a file?
      • # of master/complete copies (# of devices on which file fully exists)
      • Are devices geographically distributed or co-located? (client asks OS for location info if available)
  • Basic management of sync conflicts
    • Visual notification of conflicts (Notification, Finder/Explorer iconics)
    • Filesystem filename changes (as per Dropbox)
    • Client logfile of existing, unreconciled conflicts with basic guidance on how to clear them
  • Can flag and manage favourites and high-priority sync choices
  • Camera/photo find and sync - I use this for all photo sources now
5. Sharing folders/files with others via syncing
  • BitTorrent excels at this, but not in a secure way
  • Use the cloud service coordinator to authenticate invited users; local node authorises access to authenticated users

What does this new p2p network storage and sync world look like?

Fast forward to a world where a company like BitTorrent Sync or LogMeIn Cubby has successfully deployed a working, free + billable services p2p sync product that has the required MVP features to compete with and beat Dropbox.

The user downloads/buys the software for each device they want to sync/copy their data between - just like Dropbox.  The client includes a simple dashboard indicating sync status and data "safety".  The client can administer all participating nodes and shared folders.  Client is a gateway to buying additional services.  User can see status of all devices, summary of content all on devices, when device was last sync'ed/seen.

Offer a user a "free" version of something that feels close to what Dropbox is today, but enables the synchronisation of an unlimited amount of data across any number of personal devices.  In fact, in general the more devices you store the data on, the faster sync happens and the "safer" the data is.

Why can't Dropbox and similar existing services do this today?

They could certainly do some of it, particularly around simplified usability.

However, Dropbox and most others have a legacy business model and investment in cloud storage, not p2p.  This will hold them back from p2p.

I'm not sure Skype can pull this off either.  Now that Skype is owned by Microsoft, one can guess the internal politics between Microsoft's Skydrive and the new Skype team will slow any progress to develop this to a crawl.  Besides, Microsoft rarely demonstrates Internet-first thinking.

What that means of course is that a new player like Cubby or BitTorrent Sync may be able to slip in.

How to make money in p2p

BitTorrent never made any money on p2p.  Just a big "thanks" from the tech-savvy Interent community for the protocol that has enabled fast and resilient file sharing for years.  However, I have a penchant for actually making money as well as technical elegance, so here are some ways that might happen with a p2p sync product.

Software sales.  Clients must be purchased for each platform you want to synchronise between.  I think users would generally accept a single shot, nominal software fee for client software purchases.  Certainly after beta and pre a mass-market ramp.  Each new device type is a new software purchase.  BoxCryptor and 1Password successfully use this model.

Services.  Much to the frustration of the media rights holders, p2p (a la BitTorrent) doesn't enable a path for them to make money.  They key of course is offering truly valuable services on top of the p2p service.

Cloud based service coordination.  I think a small fee per year could be charged after the first year for data relay, authentication and similar coordination/security services.  Certainly enough to cover related operational costs.  Looks like Cubby is doing this today.

The most obvious way to generate revenue is to create value-add services on top of cloud storage and data transfer to/from cloud storage.

A backup service is the most obvious value-add offering.

Basic backup service.  User's want to be confident that their precious documents and digital photographs are backed up to a safe and secure location.  Particularly if you build in having a separate geographic location being an important criteria to receive a "Your'e safe!" rating in the status dashboard for data redundancy.  From a security perspective, cloud based backups must be a "locked box" with only the encrypted format of the file backed up to the cloud and with only the user having the key to decrypt the files (client side de/encrypt) - not like Dropbox that store all your data in their cloud servers in an unencrypted format.

Versioning service.  Deliver as an extension to backup, perhaps like Apple's Time Capsule as a growing number of people understand the Time Machine concept.  Pay as you use for frequency and size of backups/versions.

High value local application data backup services.  Some of the below are structured data files meaning backup may not just be a simple copy.  Seek and suggest high value targets for sync and backup by offering a billable backup extension:
- Photos, iPhoto DB

- Contacts
- Calendar

- Email (local)
- Video
- 1Password DB
- BoxCryptor DB
- iTunes purchased music
- Purchased software
- ...

High value social/internet backup services.  Offer each as a billable backup extension.  A little like Facebook's timeline, but platform neutral.  Use APIs to pull out social media contributions and references:
- Email (hotmail, google, ...)
- Facebook
- Twitter
- LinkedIn
- Blogger
- ...

Web-based publication service.  Provide cloud storage and controls to Internet publish and share your content.

Specialised hardware.  Offer dedicated NAS devices that directly support the p2p sync protocol or license the protocol to NAS makers (BitTorrent Sync hits this point).  Offer from 1 to 5 disk chassis.  The sync client will automatically discover any new device connected to the local network and offer to configure it for you.  Usability will be critical.

Technical Challenges

I didn't say there aren't technical challenges here.  A few of them might be:

1. How does each node determine which file is the master among all nodes?  When to sync the newest versus flag a sync conflict and rename files.  Will have to develop a strategy around inaccurate system clocks.

2. Mobile versus desktop/laptop.  How do you manage limited space and CPU access on mobile devices versus always-connected and plenty-of-horsepower desktop/laptops?  Caching and sync prioritisation is tricky in a mixed node environment.
  • User can explicitly mark high/low priority data (BitTorrent client "Transmission" has this feature) - always a high priority or just a high priority until everything in sync then back to normal priority
  • User can explicitly mark favourites to imply an on-going high priority
  • Automatically identify "hot" files through regular/frequent/recent use.  Files you are actively working on are implicitly prioritised for sync across all nodes.
  • Amazon Kindle's approach to "cloud" vs "device" location of books is a good usability model to consider and will educate mainstream users on this model
  • Always prioritise what user is asking for right now in the front of the sync work stream, ahead of what is pending (for other reasons) to be sync'ed.

Bits and pieces

There are a few other factors to consider when looking for Dropbox weaknesses.

Dropbox incurs a competitive disadvantage bourne of their very successful "share with a friend" referral model to build up membership - a whole bunch of freeloaders aren't paying for the service but still creating operational costs.  Of course, freeloaders don't cost Dropbox - the costs are borne by customers who pay for Dropbox services by paying somewhat more than the true cost of their service.  Assuming you don't sync media and just stick with documents, 5GB of free storage is room plus the refer-a-friend space bonuses can store for *a lot* of documents.

Unfortunately, Cubby appears to have copied the Dropbox business model instead of offering (e.g.) a limited duration trial and aggressive shutdown of freeloaders.  I would recommend that Cubby emphasise unlimited space free syncing for one year then charge for value-add services like data relaying and cloud backup.

Dropbox also has a higher cost by hosting their storage in AWS' S3.  They must be at a point where a dedicated in-house equivalent service with an S3 simulation wrapper around it would be cheaper.

Dropbox is still semi-technical in nature - further usability improvements are possible.

BitTorrent including their name in marketing their Sync product will be a mistake if mass-market usage is their goal.  BitTorrent file sharing has in part been inhibited from wide-spread public acceptance because of its association with illegal activity (media rights violation).  However, so long as I'm sharing my own (rightfully obtained) files… at least between my own devices… for my own use... there should be no violation.  There will have to a be a temptation with the BitTorrent approach of course to co-mingle media sharing with sync which will also inhibit mass-market acceptance.

The p2p approach to storage will take a long time to be adopted in corporate environments.  Just look at the struggle of Skype and Dropbox in the Enterprise continuing even today.


There is an emerging trend now to think "mobile first" in development.  Companies that are oriented toward browser based "traditional" Internet service consumption are at risk because a mobile first equivalent can come along and end-run them.  Similarly, p2p for network file storage and sync could easily become a disintermediating force for file sync and share services that currently think cloud first.  Is p2p sync a viable product in the "post cloud" world?  At least in this use case?

So who might pull this off?
- Although Skype *should* be the best horse to bet on, the Microsoft purchase, "Internet Last" thinking and internal politics may kill all hope
- I don't think BitTorrent Sync will be relevant to go mass market - too much baggage.

So that leaves LogMeIn Cubby can pull this off and steal market share from Dropbox and other "last year's tech" cloud storage and sync providers.  Of course, there are all the other MVP features above they need to get right as well, which is chancy.  Or perhaps some other startup or early/quiet competitor whose ramping up their operation right now…

(Postnote: A blog article specific to Bit Torrent's SyncApp was written a few days after this one.)