Are you over 18 and want to see adult content?
More Annotations
A complete backup of www.ilpost.it/2020/02/15/nise-da-silveira-chi-era/
Are you over 18 and want to see adult content?
A complete backup of football.ua/games_online/70203.html
Are you over 18 and want to see adult content?
A complete backup of www.bbc.co.uk/sport/football/51423888
Are you over 18 and want to see adult content?
Favourite Annotations
A complete backup of quartier-rouge.be
Are you over 18 and want to see adult content?
A complete backup of www.krasotkilux.ru
Are you over 18 and want to see adult content?
A complete backup of www.xhamster.desi
Are you over 18 and want to see adult content?
A complete backup of www.nudistfun.com
Are you over 18 and want to see adult content?
A complete backup of www.www.quartier-rouge.be
Are you over 18 and want to see adult content?
A complete backup of www.myhardarchive.tv
Are you over 18 and want to see adult content?
A complete backup of www.www.momshere.com
Are you over 18 and want to see adult content?
A complete backup of ugotitflauntit.com
Are you over 18 and want to see adult content?
A complete backup of www.gloryholeswallow.com
Are you over 18 and want to see adult content?
Text
WHERE WE’VE BEEN
Our last post, the review of Netflix's security model, turned out to be a complete success with thousands of visits and downloads of the full report, and it led to conversations with the engineering team at Netflix about security flaws that we identified during our study (and which they fixed very quickly --it was impressive to EIGHT-BALL TERRAINS (PART 2) Last time, I talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this: That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it doesn't take ON NETFLIX’S VIDEO STREAMING SECURITY FRAMEWORK Both kinds of players (browser-based and external) communicate with a set of Netflix servers that enforce security constraints, provide playback information, and stream the video files to the player. That communication is based on the HTTP and HTTPS protocols, and the message exchange can be observed at different times during playbackand be
SMALL-TOWN CHARM
Although websites with user generated content are all the rage these days, it often feels like their communities are too big. I face the same problems over and over when picking though content (Youtube, Amazon, Reddit, etc.): Why do I see content I don't care about? Why doI
YARS NOW AVAILABLE FOR PUBLIC USE In a previous post, I mentioned that Jay and I were working on a basic skeleton for a generic Rails app. The idea was to extract the common code that we almost always use in our applications and create a 'standard' project from which we can start new applications without having to worry about basic A DEFAULT SKELETON FOR OUR RAILS APPS I read in the Release Notes for Rails 2.3 that there will be support for application templates.In their words: Rails 2.3 incorporates Jeremy McAnally’s rg application generator. What this means is that we now have template-based application generation built right into Rails; if you have a set of plugins you include in every application (among many other use cases), you can just set up a RUBY VS. PHP FOR NON-TECHS (ALSO RAILS VS. CAKEPHP Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. Since I've been asked the same question several times before, I thought that it'd be a GOING LIGHTWEIGHT WITH SINATRA We started taking Sinatra more seriously after we heard Adam Wiggins and Blake Mizerany's talk about building lightweight web services with Sinatra and RestClient. Jay chose Sinatra to build a small geo-tagging service that he had been wanting to work on for a while, and he was very pleasantly surprised: lightweight, fast, clear, with lots STRONGLY CONNECTED COMPONENTS Last time, I talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this:. That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it RAILS | STRONGLY CONNECTED COMPONENTS Posts about rails written by Agustín Schapira. We’ve been crazy busy these last couple of weeks, with a lot of client work, and I only now find the time to write down these thoughts thatWHERE WE’VE BEEN
Our last post, the review of Netflix's security model, turned out to be a complete success with thousands of visits and downloads of the full report, and it led to conversations with the engineering team at Netflix about security flaws that we identified during our study (and which they fixed very quickly --it was impressive to EIGHT-BALL TERRAINS (PART 2) Last time, I talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this: That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it doesn't take ON NETFLIX’S VIDEO STREAMING SECURITY FRAMEWORK Both kinds of players (browser-based and external) communicate with a set of Netflix servers that enforce security constraints, provide playback information, and stream the video files to the player. That communication is based on the HTTP and HTTPS protocols, and the message exchange can be observed at different times during playbackand be
SMALL-TOWN CHARM
Although websites with user generated content are all the rage these days, it often feels like their communities are too big. I face the same problems over and over when picking though content (Youtube, Amazon, Reddit, etc.): Why do I see content I don't care about? Why doI
YARS NOW AVAILABLE FOR PUBLIC USE In a previous post, I mentioned that Jay and I were working on a basic skeleton for a generic Rails app. The idea was to extract the common code that we almost always use in our applications and create a 'standard' project from which we can start new applications without having to worry about basic A DEFAULT SKELETON FOR OUR RAILS APPS I read in the Release Notes for Rails 2.3 that there will be support for application templates.In their words: Rails 2.3 incorporates Jeremy McAnally’s rg application generator. What this means is that we now have template-based application generation built right into Rails; if you have a set of plugins you include in every application (among many other use cases), you can just set up a RUBY VS. PHP FOR NON-TECHS (ALSO RAILS VS. CAKEPHP Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. Since I've been asked the same question several times before, I thought that it'd be a GOING LIGHTWEIGHT WITH SINATRA We started taking Sinatra more seriously after we heard Adam Wiggins and Blake Mizerany's talk about building lightweight web services with Sinatra and RestClient. Jay chose Sinatra to build a small geo-tagging service that he had been wanting to work on for a while, and he was very pleasantly surprised: lightweight, fast, clear, with lots RUBY VS. PHP FOR NON-TECHS (ALSO RAILS VS. CAKEPHP Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. Since I've been asked the same question several times before, I thought that it'd be a RAILS | STRONGLY CONNECTED COMPONENTS Posts about rails written by Agustín Schapira. We’ve been crazy busy these last couple of weeks, with a lot of client work, and I only now find the time to write down these thoughts that SENDING EMAILS IN A BACKGROUND PROCESS The excellent Overstimulate the Feed has a good writeup about using the Background Job plugin to move email alerts off your controller actions and into a background job. WEB | STRONGLY CONNECTED COMPONENTS Although websites with user generated content are all the rage these days, it often feels like their communities are too big.I face the same problems over and over when picking though content (Youtube, Amazon, Reddit, etc.): DESIGN | STRONGLY CONNECTED COMPONENTS Posts about Design written by Agustín Schapira. As software developers we should spend more time studying the (relatively few) examples of systems that are well built: elegant, robust, efficient, cleanly designed and implemented. RUBY | STRONGLY CONNECTED COMPONENTS Posts about ruby written by Agustín Schapira. Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. TWITTER, EMAIL, AND SMS INTEGRATION, ALL FOR THE PRICE OF We've been crazy busy these last couple of weeks, with a lot of client work, and I only now find the time to write down these thoughts that have been on my mind for quite a while. A couple of years ago I built a time management system of my own because I couldn't find MISC | STRONGLY CONNECTED COMPONENTS Last time, I talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this:. That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it STRONGLY CONNECTED COMPONENTS Last time, I talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this:. That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it STRONGLY CONNECTED COMPONENTS I read in the Release Notes for Rails 2.3 that there will be support for application templates.In their words: Rails 2.3 incorporates Jeremy McAnally’s rg application generator. What this means is that we now have template-based application generation built right into Rails; if you have a set of plugins you include in every application (among many other use cases), you can just set up a CODE | STRONGLY CONNECTED COMPONENTS Posts about code written by Agustín Schapira. We’ve been crazy busy these last couple of weeks, with a lot of client work, and I only now find the time to write down these thoughts that RUBY | STRONGLY CONNECTED COMPONENTS Posts about ruby written by Agustín Schapira. Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. EIGHT-BALL TERRAINS (PART 1) The game of eight-ball is popular amongst amateur pool players (aka local bangers) here in the US. I end up playing a lot of eight-ball (likely because my office is next door to a bar with pool tables), and find myself looking at it from a theoretical, computer-sciency viewpoint. FSMs, optimality, minimax, variance, etc. Needless EIGHT-BALL TERRAINS (PART 2) Last time, I talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this: That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it doesn't take ON NETFLIX’S VIDEO STREAMING SECURITY FRAMEWORK Both kinds of players (browser-based and external) communicate with a set of Netflix servers that enforce security constraints, provide playback information, and stream the video files to the player. That communication is based on the HTTP and HTTPS protocols, and the message exchange can be observed at different times during playbackand be
RUBY VS. PHP FOR NON-TECHS (ALSO RAILS VS. CAKEPHP Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. Since I've been asked the same question several times before, I thought that it'd be a YARS NOW AVAILABLE FOR PUBLIC USE In a previous post, I mentioned that Jay and I were working on a basic skeleton for a generic Rails app. The idea was to extract the common code that we almost always use in our applications and create a 'standard' project from which we can start new applications without having to worry about basic GOING LIGHTWEIGHT WITH SINATRA We started taking Sinatra more seriously after we heard Adam Wiggins and Blake Mizerany's talk about building lightweight web services with Sinatra and RestClient. Jay chose Sinatra to build a small geo-tagging service that he had been wanting to work on for a while, and he was very pleasantly surprised: lightweight, fast, clear, with lots STRONGLY CONNECTED COMPONENTS Last time, I talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this:. That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it STRONGLY CONNECTED COMPONENTS I read in the Release Notes for Rails 2.3 that there will be support for application templates.In their words: Rails 2.3 incorporates Jeremy McAnally’s rg application generator. What this means is that we now have template-based application generation built right into Rails; if you have a set of plugins you include in every application (among many other use cases), you can just set up a CODE | STRONGLY CONNECTED COMPONENTS Posts about code written by Agustín Schapira. We’ve been crazy busy these last couple of weeks, with a lot of client work, and I only now find the time to write down these thoughts that RUBY | STRONGLY CONNECTED COMPONENTS Posts about ruby written by Agustín Schapira. Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. EIGHT-BALL TERRAINS (PART 1) The game of eight-ball is popular amongst amateur pool players (aka local bangers) here in the US. I end up playing a lot of eight-ball (likely because my office is next door to a bar with pool tables), and find myself looking at it from a theoretical, computer-sciency viewpoint. FSMs, optimality, minimax, variance, etc. Needless EIGHT-BALL TERRAINS (PART 2) Last time, I talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this: That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it doesn't take ON NETFLIX’S VIDEO STREAMING SECURITY FRAMEWORK Both kinds of players (browser-based and external) communicate with a set of Netflix servers that enforce security constraints, provide playback information, and stream the video files to the player. That communication is based on the HTTP and HTTPS protocols, and the message exchange can be observed at different times during playbackand be
RUBY VS. PHP FOR NON-TECHS (ALSO RAILS VS. CAKEPHP Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. Since I've been asked the same question several times before, I thought that it'd be a YARS NOW AVAILABLE FOR PUBLIC USE In a previous post, I mentioned that Jay and I were working on a basic skeleton for a generic Rails app. The idea was to extract the common code that we almost always use in our applications and create a 'standard' project from which we can start new applications without having to worry about basic GOING LIGHTWEIGHT WITH SINATRA We started taking Sinatra more seriously after we heard Adam Wiggins and Blake Mizerany's talk about building lightweight web services with Sinatra and RestClient. Jay chose Sinatra to build a small geo-tagging service that he had been wanting to work on for a while, and he was very pleasantly surprised: lightweight, fast, clear, with lotsWHERE WE’VE BEEN
Our last post, the review of Netflix's security model, turned out to be a complete success with thousands of visits and downloads of the full report, and it led to conversations with the engineering team at Netflix about security flaws that we identified during our study (and which they fixed very quickly --it was impressive to RAILS | STRONGLY CONNECTED COMPONENTS Posts about rails written by Agustín Schapira. We’ve been crazy busy these last couple of weeks, with a lot of client work, and I only now find the time to write down these thoughts that SENDING EMAILS IN A BACKGROUND PROCESS The excellent Overstimulate the Feed has a good writeup about using the Background Job plugin to move email alerts off your controller actions and into a background job. YARS NOW AVAILABLE FOR PUBLIC USE In a previous post, I mentioned that Jay and I were working on a basic skeleton for a generic Rails app.The idea was to extract the common code that we almost always use in our applications and create a ‘standard’ project from which we can start new applications without having to worry about basic and repetitive stuff like layouts, user authentication, etc. A DEFAULT SKELETON FOR OUR RAILS APPS I read in the Release Notes for Rails 2.3 that there will be support for application templates.In their words: Rails 2.3 incorporates Jeremy McAnally’s rg application generator. What this means is that we now have template-based application generation built right into Rails; if you have a set of plugins you include in every application (among many other use cases), you can just set up a RUBY | STRONGLY CONNECTED COMPONENTS Posts about ruby written by Agustín Schapira. Today I was asked by one ouf our clients why we stubbornly insist on using Ruby on Rails when most of their clients (and their hosting services) support exclusively, or preferrably, PHP. WEB | STRONGLY CONNECTED COMPONENTS Although websites with user generated content are all the rage these days, it often feels like their communities are too big.I face the same problems over and over when picking though content (Youtube, Amazon, Reddit, etc.):SMALL-TOWN CHARM
Although websites with user generated content are all the rage these days, it often feels like their communities are too big. I face the same problems over and over when picking though content (Youtube, Amazon, Reddit, etc.): Why do I see content I don't care about? Why doI
TWITTER, EMAIL, AND SMS INTEGRATION, ALL FOR THE PRICE OF We've been crazy busy these last couple of weeks, with a lot of client work, and I only now find the time to write down these thoughts that have been on my mind for quite a while. A couple of years ago I built a time management system of my own because I couldn't find STRONGLY CONNECTED COMPONENTS Notes from the folks at Pomelo.WHERE WE’VE BEEN
with one comment
Our last post, the review of Netflix’s security model,
turned out to be a complete success with thousands of visits and downloads of the full report, and it led to conversations with the engineering team at Netflix about security flaws that we identified during our study (and which they fixed very quickly –it was impressive to see how they could react in a matter of days and rollout new versions of the entire communications protocol). It’s been two months since that last post, and we have been absent without notice for a long time. The reason: we’ve been very busy! In addition to lots of client work, with Comcast and Pearson Education, we have been busy working on two projects for the Pomelo labs , which have also turned out to be quitesuccessful:
* You may have already heard about the first one, Happn.in . If not, be sure to check it out. Jay has been working like crazy on it for a couple of months, and it shows. Happn.in keeps track of what people are twittering about in your city: it collects and aggregates popular phrases used on Twitter within 20 miles of major cities; the five most popular phrases each hour are posted to this site, and are tweeted three times a day to the happn.in Twitter account for each city. As I write this, there are about 98,587 people following happn.in in 63 cities!. The cool thing is that you can sign up to be the sponsor a the list for any city, for any period of time, and get your site’s URL seen by thousands of people on Twitter. If you are interested in seeing how this all works, check out the site’s About page, and if you’re a researcher and want to get your hands on this kind of data, feel free to contact Jay at jay@happn.in. * The other project is something I have been working on with Anita, my wife. It’s called FromABirdie , and in a way it targets the other end of the writing spectrum. It’s a site where people can create an “_album_” and invite others to write letters for a common friend. The letters are kept in a private album that only the recipient can access and, on a pre-specified day, we send him or her an email to announce the surprise that his/her friends have created. It started as a personal website that I built for my wife’s XXth birthday, and since then it has evolved into a public service. We have seen it used for birthdays, graduations, Mother’s Day and Father’s Day, for a colleague that was moving, and just for fun. It’s been very rewarding in many ways, and quite surprising to see how people write long, detailed, and deep letters to their friends, something that I thought didn’t happen anymore in this age of Facebook, cell phones, and instant communications. By the way, it’s free, so feel free to use it!.
These have been two wonderful months, full of fun stuff. We have also been lucky to collaborate with the great Matt Latkiewicz,
the brilliant design mind behind Happn.in , and I’ve been working with two wonderful undergraduate students from UMass: Aaron Packard, who is helping me with the Ruby on Rails engine behind FromABirdie , and Ann Wong, who is doing great work for Happn.in visualizations using Processing . It’s been a realpleasure.
The plan now is to continue working on these two projects (nights and weekends, after we’re done with client work!) and to try to see them grow. Oh, and I’ve also been invited to attend the “_Seminar on Innovative Approaches to Turn Statistics into Knowledge_“,
organized by the World Bank, the Census Bureau, and the Organization for Economic Co-operation and Development, and I am looking forward to it. Lots of things. We promise to keep you posted more frequently!Advertisements
Report this ad
Report this ad
Written by Agustín Schapira June 12, 2009 at 12:22 pm Posted in Uncategorized ON NETFLIX’S VIDEO STREAMING SECURITY FRAMEWORKwith 6 comments
As software developers we should spend more time studying the (relatively few) examples of systems that are well built: elegant, robust, efficient, cleanly designed and implemented. It is not only a wonderful way to learn from the masters, but also a very gratifying experience and a source of intellectual pleasure and stimulation. Last week we had some time on our hands and decided to spend it poking and studying the security model behind Netflix’s video-on-demand service (“Watch Instantly”). We wanted to learn about the constraints they enforce, the mechanisms they use, and the ways in which those constraints and mechanisms affect performance and the user experience. We were not surprised to discover that Netflix’s security framework is as robust and well designed as the developer APIs that they provide. We share here what we learned, hoping that others can benefit from it as well. (But we also want to make it clear that what follows is simply our understanding of how things work, which does not necessarily map one-to-one to they way things actually work. We were meticulous in our study, but because we were basically reverse-engineering their process, there certainly will be some misunderstandings and things left out). The full report is available in PDF format,
along with a text file with a complete dump of the messages exchanged between the Netflix player and the Netflix servers. We include below a copy of the first two sections of the report, describing the methodology and providing a general overview of the problem and thesolution.
A BRIEF INTRODUCTION AND METHODOLOGY Netflix’s video-on-demand service offers its subscribers the possibility to watch over 12,000 titles online through their Internet connection. It works on a PC/Mac with a browser-based player created by Netflix (written in Microsoft’s Silverlight), or via a Netflix-ready external player such as a Roku Box, an Xbox 360, the Samsung Blu-ray player, or any other Netflix-enabled device. Both kinds of players (browser-based and external) communicate with a set of Netflix servers that enforce security constraints, provide playback information, and stream the video files to the player. That communication is based on the HTTP and HTTPS protocols, and the message exchange can be observed at different times during playback and be used to make inferences about the kinds of mechanisms that Netflix must have in place in order to provide a secure videoexperience.
Our analysis was centered on the browser-based playback experience. We created a new Netflix account and observed the message exchange using Firefox and Tamper Data, a Firefox
plug-in that traps HTTP and HTTPS requests and responses to/from the server and displays their content, headers, and other usefulinformation.
We also briefly studied the behavior of external players. Precisely because those interactions occur outside of a web browser, we were unable to use any high-level tools such as Tamper Data to analyze the message exchanges. Instead, we had to rely on Unix packet sniffers (mainly tcpdump and Wireshark), which provide less detailed information and, in particular, make it much harder to decrypt HTTPS messages (especially since in these cases the key is locked inside the device!). Even though we were unable to analyze the messages as deeply as for the browser-based players, we still were able to detect many similarities and a few differences between browser-based and non-browser based players, and we detail them at the end of thisreport.
SECURITY REQUIREMENTS, CONTROL MECHANISMS, AND GENERAL ARCHITECTURE Netflix concerns, from a security point of view, are: * Only users with the correct (unlimited) plan can access content * Users don’t share their account information with others * Videos are only accessible from within the US, due to licensingrestrictions
* Video content cannot be redistributed and played at a later time Those restrictions are enforced by a set of coordinated mechanisms: * Request user authentication before playing the video * Allow a maximum of six playback devices (browsers, Roku, etc.) peraccount
* Reject requests that come from IP addresses outside of the US range * Encrypt the video content * Provide unique decryption keys per movie and device The security checks take place over the duration of the playback experience, and are delegated to several different Netflix components and servers, each with distinct responsibilities. The following diagram shows the steps required to play a video on Netflix (slightly out of order, for clarity), the different servers involved in each process, and their respective responsibilities. Security responsibilities are also distributed on the client side, between the browser (which stores and sends cookies), the Netflix player (which follows the playback and security protocols described above), the Microsoft Silverlight player in which the player is implemented (which prevents cross-site scripting and makes sure that the player only talks to Netflix servers), and a proprietary Microsoft DRM component (which gets an individualized key for each player and coordinates with the License server to acquire keys that are unique to the movie and the player). This architecture provides several advantages: * Because all playback devices are registered and associated with an account, Netflix can prevent abusive users from sharing their credentials with others; if they do so, their account will quickly run out of allowed devices. (Note that here a device corresponds to a user on a PC/Mac, or a Roku, Blu-ray, etc. external device. Two browsers on the same machine/user are one device, but two users on the same machine are considered to be two separate devices.) * The encryption of video files ensures that only players with knowledge of the key and the decryption algorithm (proprietary Microsoft technology) can view them. * If one of those decryption keys is compromised then the security threat is minor, because the keys are individualized and can only beused by one player.
* Thanks to this distribution of tasks, different servers and client components can focus on the particular constraints that they have to enforce, and assume that other constraints are checked somewhere else. At the same time, and because information is shared between the different servers (through cookies and extra headers in the HTTP requests), security on each of these components can be tightened as desired, if/when Netflix detects abuses or security breaches at aparticular point.
* Because all communication takes place using well known HTTP and HTTPS protocols, Netflix has been able to easily provide their video-on-demand service to non-browser devices (such as Roku or Xbox 360) simply by implementing a player on the client side. All these non-browser players communicate with the same servers and follow more or less the same protocol as the browser-based player. * Since all authentication and authorization checks are performed before playback begins, and since the content of files is encrypted, the streaming of the video files can be easily delegated to CDNs, increasing delivery speed without compromising security. * Finally, because the playback experience is controlled through Netflix’s proprietary player, the enforcement of security constraints can be hidden almost completely from users. All they are required to do is log into the site using their account information. The registration and authentication of the device, and the acquisition of playback licenses, all happen behind the scenes. For a step by step walkthrough of a typical playback session, you can download the full report (18 pages, skip the first two sections), and also get a text file with a complete dump of the messages exchanged (remove .doc extension, which WordPress forced us to use). Written by Agustín Schapira April 15, 2009 at 8:00 amPosted in Design ,
security , Systems
SMALL-TOWN CHARM
leave a comment »
Although websites with user generated content are all the rage these days, it often feels like their communities are _too big_. I face the same problems over and over when picking though content (Youtube, Amazon, Reddit, etc.): * Why do I see content I don’t care about? * Why do I have to weed through comments or reviews from people who don’t think like me? * What use is a four-star rating over 25,000 reviews if most of the reviewers are morons? The cause of these problems is understandable; websites have built up all this content, and they want to show it off. A site that boasts a large number of users or reviewers is more likely to continue growing, as people consider it a reliable source for content. But how can we manage large communities (and related content) such that they still maintain small-town charm? Specifically, how can we match users and content? I’ll describe one such solution: INCREASING THE USEFULNESS OF CONTENT-HEAVY WEBSITES BY MAINTAINING CLUSTERS OFSIMILAR PEOPLE.
These thoughts were inspired by this article. It’s long and
great; read it. To summarize, the article proposes some mechanisms for maintaining community within social bookmarkingsites. Many of the
proposed ideas can be generalized to all web communities. The Internet has supported community for a long time. Here’s a general timeline of Internet communities I’ve been involved in over the years (although mostly as a lurker): VAX Notes> Usenet
> various Listservs
> Slashdot
> Kuro5hin > Reddit> Hacker News . I
generally stick around until the community grows too large, then moveon to a new site.
There’s all sortsof writing
about Internet
communities; what makes them thrive or fail, and how they relate to real-world anthropology. More than these aspects, I’m interested in how we can provide a good experience for users, which in turn can _promote the formation of community_. To illustrate the issue, let’s look at recipe websites, since I can’t seem to find one I like (although Cookthinkis pretty nice).
Consider somebody who stumbles on a recipe website and wants to find something for dinner. They type in (or click on) ‘pasta’, and see a slew of results. Since there are many styles of cooking, it’s quite likely that many of the results are useless to that person (too simple, too complicated, too unhealty, etc.). Although we don’t know anything about the user yet, we want these results to become more useful in the future. Most commonly, the recipes are ordered by popularity or date, and the visitor can sort or filter the results by criteria such as cooking time or ingredients. In other cases, users can join pre-established communities (e.g. subreddits ), and content is dictated by these group memberships. Sometimes, related content can be displayed (like Amazon’s _Customers with Similar Searches Purchased…_). But how can we display appropriate content without the user doing anything? Perhaps on their first search, we can’t. But as soon as they start clicking, we’re learning about the user. We’re learning which recipes they like, and we’re learning how the user relates to all of those recipes’ attributes. We’re learning about the recipes and attributes they don’t care about. More importantly (in the interest of promoting community) we’re discovering how this user _relates to other users_. Suppose, after a few clicks, we assign the person to some subgroup of similar users (we maintain a user graph with edge weights that represent community proximity). It’s transparent to the user; they have no idea this happens. From now on, we display content, comments and reviews only from closely related people, perhaps with some more distant content to fill in gaps. All that other stored content? Perhaps it’s accessible through deeper searches, or perhaps it’s not accessible at all. People don’t like being presented with an immense number of results to sift through; just the good ones. It seems to work for In-N-Out.
The key to this scheme is some metric to evaluate the happiness or satisfaction of users. Sometimes, site success can be measured by the length and quality of discussions (some users may prefer one-word conversations while others prefer longer, more involved responses). Perhaps the happiness of users of a cooking site can be measured by a constant or increasing frequency of use. The graph (and its related subgroups) can be shuffled or destroyed based on the percieved happiness of each user. Perhaps the site itself (the attributes used to generate the user graph, or the metrics used for happiness) can evolve to produce higher quality groups. This idea isn’t about tacking social networking aspects on to existing sites. It’s about transparent mechanisms with which sites can better present content and promote community. Though I’m not a social scientist, I’ll bet that when users consistently see the same names on comments or reviews, they’ll be more likely to forge relationships. And isn’t that what we’re all about? Written by Jay Boice March 31, 2009 at 3:27 pmPosted in web
TWITTER, EMAIL, AND SMS INTEGRATION, ALL FOR THE PRICE OF ONEleave a comment »
We’ve been crazy busy these last couple of weeks, with a lot of client work, and I only now find the time to write down these thoughts that have been on my mind for quite a while. A couple of years ago I built a time management system of my own because I couldn’t find anything out there with the functionality that I needed. On the surface this system (which I lovingly named Heraclitus, in honor of the greek philosopher who made it impossible for us to bathe in the same river twice ) is just a regular calendar & to-do list combo. What really makes Heraclitus so special for me, though, is the fact that instead of just checking things off the list and removing them from the database, Heraclitus keeps logs of everything I’ve done, along with notes associated with each task. This means that I can go back later and find, for example, the exact list of commands that I happened to use on April 15th, 2007 to create a new AMI for EC2 (and it’s all searchable, of course).I use Heraclitus constantly, not just as a way to organize my day, but also as a knowledge base of everything that I’ve done, every article I’ve read, every client conversation I’ve had, every technical problem I’ve encountered and the solutions I’ve found, etc. etc.etc.
Heraclitus lives and runs on my laptop, which goes with me almost everywhere. There are, however, times when I am away from the machine, and of course I’d still like to be able to schedule a task within the system, or record an idea, or check something off. Sometimes I’d like to do that from my cell phone, other times via the more convenient email, and other times even via Twitter (for example to log a little remark about a interesting tweet that I’ve just read without having to leave Twitterrific). That functionality would be great to have but, needless to say, I’ve never had the time nor the desire to write all the three necessary interfaces to support it. But the other day I realized what should have been obvious all along. Because Twitter can talk to cell phones and to email (via TwitterMail ), all I had to do was build an interface to Twitter and, for free, I’d get the two other interfaces. And so I prompty added a background runner to my Heraclitus system, a simple script that periodically monitors direct messages to a Twitter account that I created for this purpose. Thanks to rest_clientand hpricot
, the code is really simple:require 'rubygems'
require 'rest_client'require 'hpricot'
last_direct_id = TwitterCheck.find_by_name('last_direct_id').valuewhile true do
# Wrap in a begin/rescue, because Twitter *will* failbegin
url = 'http://account:pwd@twitter.com/direct_messages.xml?since_id=#{last_direct_id}' dms = Hpricot( RestClient.get(url) (dms/'direct_message').each do |m| sender_name = (m/'sender/screen_name').inner_html next unless sender_name == ''# Get the text
id = (m/'/id').inner_html text = (m/'text').inner_html # If this fails, don't record the changebegin
date, descr = text.split(/ /, 3) Task.create(:name => descr, :on_day => date) TwitterCheck.update_all("value = #{id}", "name = 'last_direct_id'") last_direct_id = idrescue
ActionController::Base.logger.warn('An error occurred: #{$!}')end
end
rescue
ActionController::Base.logger.warn('An error occurred: #{$!}')end
sleep 240
end
The script uses a database table to record that last message that it received, and every two minutes it requests from Twitter the NEW direct messages for that Heraclitus account. If there are any, it verifies that the message was sent by me (so that other people cannot spam me), and then it parses the message according to my rules and adds it to my schedule. So now I can go to Twitter and send a message to my account like this: d myheraclitus 04-15 Pay Taxes @ 2:30 and in a couple of minutes it’s listed on my to-do. The cool thing is that I can also text the same message from my cell phone (using shortcuts so I don’t have to type all that text), and I can email it to a TwitterMail account that then forwards it to Twitter, and from there to my Heraclitus. All three interfaces for the price of one! Written by Agustín Schapira March 29, 2009 at 3:26 pm Posted in code , rails, recipes
, web
EIGHT-BALL TERRAINS (PART 2)leave a comment »
Last time
, I
talked about how we can visualize our shots during a game of pool in a three-dimensional terrain like this: That diagram, though, is a simplified method of choosing a shot. It includes only our preference for the location of the cue ball after the shot; there are other factors it doesn’t take into consideration. Here I’ll talk about several of them, including:* Our opponent
* Shot difficulty
* Our skill level
When we look at the terrain of possible locations for the cue ball, notice that we’ve made one large assumption; that we will _pocket the object ball_ and have the next turn. Of course, we all miss shots (some people/shots more often than others), so making this assumption ignores the possibility that our opponent will shoot next. Occasionally, when players don’t have an easy shot, they play a _safety_; a shot where they don’t attempt to pocket a ball in order to leave the opponent a very difficult shot. In fact, we can visualize the range of safeties with the same type of terrain: In this diagram, the height of the terrain represents our opponent’s preference for the cue ball to land in that spot (assuming we miss our shot). In this situation (again, this diagram isn’t necessarily realistic, but used to illustrate an idea), we’ll assume that our opponent wants us to leave the cue ball so that he has a shot on the 12 (purple striped) ball. This means that if we were to play a safety in this situation, we want to miss our shot, and leave the cue ball anywhere _except_ the peak at the end of the table. And most pool players, when taking a shot, are trying to accomplish one of the two objectives shown in the prior two diagrams. They’re either playing offensively by trying to make a ball, or they’re playing defensively with a safety. But that doesn’t _have_ to be the case; sometimes we can take a shot that, if we make it, will leave us an excellent next shot, and if we miss it, will leave the opponent a very difficult shot. We can visualize this with the composition (difference) of the two terrains: The peak that was previously at the end of the table in our offensive terrain matched our opponent’s peak, so our preference for that shot may now be lower. The peaks in the new diagram are those locations where we will have a fairly easy next shot _if we make the object ball_, and our opponent will have a difficult shot _if we miss_. This terrain still isn’t accurate; it should also take as input the difficulty of the shot. This can can be done by weighting the two terrains by shot difficulty when composing them — if the shot is simple, and we make it 80% of the time, we should weight the offensive terrain more heavily than the defensive terrain. If a shot is more difficult, we should weight the defensive terrain more heavily. Recognizing the difficulty of a shot is hard because it has a psychological aspect. We must accurately gauge our own skill level. If we overestimate our abilities, we’ll weight the terrains incorrectly, which will lead us to the wrong decision. Humility isimportant.
And we could go on. Remember that this terrain looks only at the cue ball, and ignores the other balls on the table. This is natural for a pool player; in general, we try to hit as few balls as possible (unless we have a need to move one into a better position.) The diagram should probably be in n+2 dimensions, where n is the number of balls on the table. That would be tough to draw. What use is all this? I’m sure we could write a nice little pool-playing bot that takes all these aspects of shots, and some known variance (its current skill level) into consideration, and minimaxesthem.
A pool bot is pretty useless (I don’t see us carrying around little terrain generators until we have holographic iPhones or something), so here are some things I’ve learned from thinking about the game inthis way:
* You don’t always have to decide between making a shot and playing a safety; sometimes you can do both * Humility is important; recognizing you’re not good enough to make a shot enables you to make a better decision * Visualizing your terrain, and possible opponent terrains actuallyhelps your game
* Don’t try to talk about this stuff at the bar Written by Jay Boice March 18, 2009 at 3:25 pmPosted in misc
EIGHT-BALL TERRAINS (PART 1)with one comment
The game of eight-ball is popular amongst amateur pool players (aka local bangers)
here in the US. I end up playing a lot of eight-ball (likely because my office is next door to a bar with pool tables), and find myself looking at it from a theoretical, computer-sciency viewpoint. FSMs, optimality, minimax, variance, etc. Needless to say, this doesn’t hook a lot of listeners at the pool hall, so I’ve learned to keep my mouth shut. Instead, I thought I’d write down some of my ideas here. The general goal for each turn in eight-ball is to hit the cue ball so that it knocks an object ball into a pocket (if that didn’t make sense, read the rules.) Each
shot is quite different, and the optimal shot depends on many factors, the most important of which is the shape of the table, or how the balls are arranged. It’s not always a clear-cut matter of making the next shot — sometimes we _don’t_ want the object ball to go in the pocket, or sometimes we want _two_ object balls to go in pockets. There are two components of pool that are equally important: * Knowing what shot to take next * Executing that shot The second component can’t be learned without a lot of practice, but the first can be learned without even picking up a pool cue, and that’s the part I find fascinating. From one perspective, we can look at a turn as a dual-stated affair; the state (shape) of the table before our shot, and the state of the table after our shot. An optimal shot would involve sinking all our balls, with the 8 ball landing next to a pocket, and the cue ball nearby (so we can make the 8 ball on the next shot). Usually, though, the state of the table after a shot is similar to its prior state, although the cue ball has moved, and perhaps another ball has gone in a pocket. There are _many_ possible states, mainly limited by how hard we can hit the cue ball, the spin we can put on it, the speed of the table, and the laws of physics. But how do we choose one of these states to aim for? Generally, a turn is simplified into two immediate objectives: * Make one of our balls * Leave the cue ball in a good spot to make the next ball For example, if we are solids in the image below, we’d like to pocket the 5 (orange) ball, and leave ourselves a shot on the 2 (blue) ball. A word of warning to pool players about these diagrams: they’re simplified — I just whipped them up to illustrate someideas.
A good spot to leave the cue ball can be many places on the table (some subset thereof), and is possibly a large area of the table we can aim for. Often there are multiple such zones, some larger than others, and some more difficult to aim at. We can actually visualize these areas in three dimensions, like this: The three-dimensional terrain illustrates possible locations of the cue ball after our shot, and its height represents our preference for that location. The steeper peak in the illustration may make the next shot simple, but is difficult to aim at. The wider, rolling hill doesn’t make for such an easy next shot, but is much easier to aimfor.
This is where the decision gets more difficult. The terrain we generated is simplified; its only input is our preference for the final location of the cue ball. Other factors, such as shot difficulty, should be included in the computation of the terrain, which may alter our final shot decision. Next time: Humility and our opponent (i.e. variance and terraincomposition)
Written by Jay Boice March 15, 2009 at 3:24 pmPosted in misc
YARS NOW AVAILABLE FOR PUBLIC USEleave a comment »
In a previous post
,
I mentioned that Jay and I were working on a basic skeleton for a generic Rails app. The idea was to extract the common code that we almost always use in our applications and create a ‘standard’ project from which we can start new applications without having to worry about basic and repetitive stuff like layouts, user authentication, etc. Well, the skeleton is now ready, and available for public use from GitHub. YARS (YARS is Another Rails Skeleton) comes pre-initialized with a convenient set of plugins, libraries, controllers & views. It also contains a modified scaffold generator that creates much simpler (non-RESTful!) controllers and views. To use it, you simply clone the repository and * Edit config/database.yml * Edit ActionMailer, ExceptionNotifier, and SYSTEM_NAME setups in config/environment.rb * Run rake db:migrate * If you want, edit public/stylesheets/application.css (but not blueprint, so we can upgrade it later if necessary) And then you’re ready to go!PACKAGES INCLUDED
PLUGINS
* acts_as_taggable_on_steroids* file_column
* will_paginate
* exception_notification* action_mailer_tls
* restful_authentication* role_requirement
* better_nested_set
* ssl_requirement
* BackgroundJob (BJ) * assert_valid_markupCSS
* blueprint
ADDED/CHANGED BEHAVIORSSCAFFOLD GENERATOR
$ ruby script/generate scaffold Model field1:type field2:type * Generates simple controller with four methods: _index_, _new_,_edit_, _delete_
* _new_ and _edit_ handle GET requests to display the form, and POST requests to process it * All actions redirect back to _index_ * Two simple views: _edit_, and _index_ RESTFUL_AUTHENTICATION * Modified so that users are _only_ identified by _email_ and _password_ (not by _username / login_). * Added method to _change_password_ (/change_password) * Added method to retrieve forgotten password (/amnesia). The method allows the user to specify a new password, which is stored but doesn’t automatically replace the current password. An email is sent to the user’s address with a link to confirm the change. When the link is followed, the newly set password replaced the old one, and the user is redirected again to the login page. * Sends link to confirm password change in the background, usingBackgroundJob (BJ)
. See
jobs/email_changed_pwd.rb * Requires SSL for login and signupROLES
* Added default roles _admin_ and _user_ * When a new account is created via Restful_Authentication, it makes it by default be a _user_SSL_REQUIREMENT
* Use by specifying ssl_required in your controller (and specify actions if necessary) * Turned off by default in development mode (see ApplicationController)LAYOUT
* Uses blueprint to create basic grid. * Header has _SYSTEM_NAME_ (defined in environment.rb), and links to _Log In/Out_ and _Sign Up_ * Footer shows _AUTHOR_NAME_ with link to _AUTHOR_URL_ (defined inenvironment.rb)
ACTIONMAILER
* Setup to use action_mailer_tls, so that it
sends email via GMail. * See UsersController/amnesia and jobs/email_changed_pwd.rb to get an idea of how to send email in the background.See the DETAILS
file for more information about the setup of each plugin, and let us know if you have any questions. Of course, feel free to fork and extend in your own way! Written by Agustín Schapira March 12, 2009 at 3:24 pm Posted in code , railsOlder Entries
ABOUT
This blog is maintained by Pomelo , a software consulting firm in Massachusetts. We are mostly interested in software design and development, scalable architectures, geo-located systems, API design, and cloud computing. We generally write about Ruby, Rails & friends, Java, and Flex, but you might also find here posts about music and film, urbanism, art, literature, and politics. COLLECTED RAILS RECIPESReport this ad
Details
Copyright © 2024 ArchiveBay.com. All rights reserved. Terms of Use | Privacy Policy | DMCA | 2021 | Feedback | Advertising | RSS 2.0