Why do so many people (and companies) pretend that app backends are just (logic-less) data stores?
I can definitely see the argument that some mobile games only need to store/sync some basic state information on behalf of the user, but I just don't see it as being that simple for most other apps.
I suppose this datastore-only backend model works for apps with users who are completely isolated from each other, but any time you want to allow users to interact with each other (or provide rules for how their resources interact with the world outside their little "user bubble"), you need business logic that resides somewhere other than the app; otherwise, your business logic (which may affect more than just the authenticated user) is in an untrusted environment. Not only that, but there are huge scalability problems associated with using a logic-less backend storage model if you want to do anything that involves more than just a single user's data.
Even for games, something as simple as providing a leaderboard is impossible (or extremely impractical) with this model. If you have no server-side logic to perform aggregate operations over multiple users, the only other option you'd have is to basically grab all the data and perform those operation in your app, which is absurd.
Backend development, server management, and deployment can be hard (though many PaaS solutions are making it easier than ever), but personally I'd never recommend building an app with a logic-less backend data store like this. Even if you don't think you'll need it today, in most cases you're going to need logic on the backend eventually. If you've started with this, you're just creating a lot more work for yourself when you finally do need to implement a feature that crosses the line from isolated-users-only to users-that-can-interact (or even just obtaining aggregate user data for business intelligence purposes).
Completely agree! At post.fm we're trying to design an email client which will work offline, and synchronize data when online (IMAP isn't sufficient for our needs) - Exchange style. We've thought long and hard about various solutions, and have tried syncing logic-less document-stores such as this. As you say - this doesn't really work, and there's definitely no way to handle anything remotely complex like transactions. In our case "app data" is effectively all your mail and more, and we can't expect each client to have a local copy of everything, meaning partial syncing of a collection is important for us (not something any solution out here seems to handle).
The solution we've come up with, which we'll hopefully get to open-source one day, is to model state over time as a persistent "event log", which means instead of syncing documents we sync events that clients perform, replaying events instead of document-update statements. This approach is infinitely flexible I believe, but requires more work if your clients are written in different languages (we use JS throughout so that isn't an issue).
I don't think anyone suggests that it's that simple. This entire space is young and evolving quickly. Identity and data sync is a good place to start, though.
For your example of a leaderboard, Parse has Cloud Code. It's designed to do exactly what you describe -- define a function that sits on Parse's server, handles client queries against your data, and returns the relevant data. Their example is for computing the average rating for a movie from your users' reviews: https://parse.com/docs/cloud_code_guide#functions
Amazon, Google, and Microsoft aren't there yet, but I can only imagine it's on their roadmap. And Parse's implementation only works for fairly easy stuff. But I think things will look very different in 5 years.
Parse's solution is a great start as you say, but I can see myself running into the limits of that approach fairly quickly.
And, as you say, if these solutions mature, we might have some very powerful stuff in this space in a few years... maybe almost as powerful as just having your own server-side code running on some backend servers ;) But at that point, we're right back to where we are with traditional/custom backend systems being run on cloud server providers (which is not a bad thing!).
It just kind of feels like we're trying to re:Invent the wheel. You can offer these kinds of "a la carte" services all day long, and they're nice (on the surface) as long as they play nicely with each other in an open ecosystem - e.g., "let's grab this identity provider, that storage solution, this analytics engine, etc." It might look similar to Heroku's "add-ons", but with components that perform more specific functions. I just don't see that openness coming from Amazon, Google, etc. as they will all want to lock people into their services (which is understandable from a business standpoint). And unfortunately, anyone who buys into partial (and proprietary) solutions in the meantime are - in my opinion - just shooting themselves in the foot by limiting the use of their data.
Even in a perfect world with a vast marketplace of plug-and-play, interoperable cloud components, I would ask: is the "soul" of your app's code really captured in a conglomeration of components like this? If you have a need for a backend with custom logic of any kind, I think you're probably doing something unique (hopefully anyway). If you're going to need some real, unique business logic that can't be offered as an abstract cloud component, then I strongly suspect you're going to want more control over all of the backend. Even Parse's approach ends up looking more and more like a traditional custom backend system as you add more power to the "Cloud Code" approach.
As a programmer, you can often find libraries and drop-in components that accomplish 80% of what you need, so you write some custom code to "glue" those components together. That custom code usually contains business logic that really couldn't be abstracted or offered as a drop-in component. Where does that custom code live in the "perfect" ecosystem of plug-and-play cloud server components? How do you control the thing which makes your app/backend business logic unique? I don't think it's going to be in the client/mobile app as some people tend to think.
Maybe there's a good solution that combines these kinds of components with custom (and flexible) server-side code - I just haven't seen what it looks like yet. If I tried to use this stuff today, I'd be kicking myself tomorrow as I figure out that it's a pain to juggle data between components that I don't control and components that I do control.
I was commenting to a friend that the limited functionality in these types of services have very steep edges. It's great when you're in the middle, but trying to do something a little bit different can lead to long, poor, difficult code as you try to work around the constraint.
I think we'll start seeing more "opinionated" platforms, similar to what Rails and others have done for web development. Parsing query strings, managing database connections, and wiring your own user reg code are all distant memories for most web developers most of the time today.
We might see something similar here, where groups of common functions or services are stacked together to form a unit that works well together and covers some set of use cases very well. LAMP comes to mind as a comparison, and you can definitely use SQL Server instead of MySQL, but it probably won't work as well when it comes down to drivers, libraries, recruiting, etc.
These stacks will differ as well to serve a broad range of customers. I cringe a bit when I see Amazon EC2 comparisons to Hetzner or Digital Ocean because they are serving quite a different set of customers with different needs. To think that all hosts should end up standardizing on the same units would leave many customers unserved. It's easy for innovation to be called vendor lock-in, but it's hard to compete or innovate on heavily commoditized products/services.
Parse/Cognito/etc allow you to quickly validate an idea that may or may not work out without having to invest in a backend engineer and a bunch of backend infrastructure. If it doesn't work, you've saved a bunch of time and money. If it does work, then you can start worrying about how to scale it and add more features. No piece of software has ever launched without technical debt, and there's much worse technical debt to take on than an outsourced backend.
instead of having to worry about building and managing a backend solution to handle identity management, network state, storage, and sync.
From my reading of this, this wouldn't in any way obviate the need for a backend system of your own. It just does the identity management, network state, storage and sync parts for you. Your own business logic on your own backend can use this to jump start you.
> Building the backend... is a lot of work. You have to build it, deploy it, and manage the infrastructure that it runs on."
To me, it certainly sounds like they're trying to market this as an alternative to having your own backend system.
And to be perfectly honest, if you do use your own backend system for anything, authentication and simple data storage are not hard to add - especially if you're going to use a SSO service like Amazon, Facebook, or Google (which Cognito uses). There are drop-in solutions for most of those things that require little more than a couple lines of app code to authenticate users. I really just have a hard time seeing the value in what Cognito offers.
Also, assuming you do just use Cognito for the auth and storage/sync parts, how do you plan to use that data in the rest of your backend system? Are you going to pull down all that data from Amazon's servers just to do what you need with it (will they even offer that ability)? If so, how do you make sure the rest of your backend infrastructure stays in sync with Amazon's user data? It just sounds like it creates a lot more complexity than actually solving anything (except in the most basic scenarios without a need for a separate backend system).
I can definitely see the argument that some mobile games only need to store/sync some basic state information on behalf of the user, but I just don't see it as being that simple for most other apps.
I suppose this datastore-only backend model works for apps with users who are completely isolated from each other, but any time you want to allow users to interact with each other (or provide rules for how their resources interact with the world outside their little "user bubble"), you need business logic that resides somewhere other than the app; otherwise, your business logic (which may affect more than just the authenticated user) is in an untrusted environment. Not only that, but there are huge scalability problems associated with using a logic-less backend storage model if you want to do anything that involves more than just a single user's data.
Even for games, something as simple as providing a leaderboard is impossible (or extremely impractical) with this model. If you have no server-side logic to perform aggregate operations over multiple users, the only other option you'd have is to basically grab all the data and perform those operation in your app, which is absurd.
Backend development, server management, and deployment can be hard (though many PaaS solutions are making it easier than ever), but personally I'd never recommend building an app with a logic-less backend data store like this. Even if you don't think you'll need it today, in most cases you're going to need logic on the backend eventually. If you've started with this, you're just creating a lot more work for yourself when you finally do need to implement a feature that crosses the line from isolated-users-only to users-that-can-interact (or even just obtaining aggregate user data for business intelligence purposes).