There are some pretty legitimate concerns with Ember and large data sets right now as the treatment of those datasets tends to be fairly naive and linear. There are several people, myself included, who are working on ways to address that problem--but modifying a large collection, at the moment, can trigger the entire collection to re-render which can be extremely expensive.
FWIW, I'm not an expert on Ember.js or Backbone; just someone trying his hand at hacking the Ember.js code to work more effectively with collections (since some of the use cases I have would be impossible using the current methods of bindings and collections).
It really depends on your environment and the complexity of the rendering tied to your collection. For example, on a fast desktop-class system with a collection that renders one element with 1 property, you can probably have hundreds (or maybe even thousands) of items in the collection before you start to notice slowdowns when the collections update. Working on lower-end and mobile platforms, those numbers change. Similarly, if you rendering is complex, it will take more time as well. All that said, there's really no "magic number"
There are definite workarounds such as setting up paging--but the simple approach of using Ember.js bindings to bind a collection to a template will fall to pieces as that collection grows.
FWIW, I'm not an expert on Ember.js or Backbone; just someone trying his hand at hacking the Ember.js code to work more effectively with collections (since some of the use cases I have would be impossible using the current methods of bindings and collections).