Ingesting the Facebook stream using Feeds and OAuth 2.0
Sun, 11/14/2010 - 04:09 — infojunkie
Now that Twitter and Feeds are buddies, time to move to other data sources. Next up: Facebook. Using trusty Feeds and friends, I was able to ingest my own Facebook home feed. Here's how to replicate this:
For the impatient, attached is a feature that should get you set up quickly. You'll need the following modules:
- Feeds (Latest dev)
- Feeds JSONPath parser (make sure to install the needed JSONPath library as per the instructions on the module page.)
- Feeds OAuth (Latest dev)
- php-proauth library that you install inside feeds_oauth module as such:
svn checkout http://php-proauth.googlecode.com/svn/trunk/ php-proauth-read-only The idea behind the setup is to create a Feeds pipeline that:
- Fetches the given resource URL (a Facebook Graph API URL) using Feeds OAuth 2.0 Fetcher. This fetcher checks for OAuth 2.0 access token for the current user and performs authorization if it's not found. It alerts the user to missing access tokens during feed creation.
- Parses the result using Feeds JSONPath Parser, since Facebook Graph API uses JSON.
- Maps the result to a node using the standard Node Processor.
To create a new feed, you need to create a node of type Facebook feed. You need to configure it a bit because Feeds JSONPath Parser is still rough around the edges:
- Context: $.data.*
- title: message
- guid: id
- body: description
- URL: your Facebook Graph API URL, e.g. https://graph.facebook.com/me/home
Obviously there's lots to be done in the mapping, but you get the general idea. Enjoy!
| Attachment | Size |
|---|---|
| facebook_feed-6.x-1.0.tar | 16.5 KB |
"There is a module for that" by Karim Ratib is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.
Drupal is a registered trademark of Dries Buytaert.
Comments
This module is exactly what I needed. I had a query though. Will it be able to trigger on its own after lets say, every 1 hour? I want to import feeds from many facebook pages and I want to do it programatically, i.e. I do not want to click the import button everytime. I am okay with saving my facebook userid and password in DB if that is all that'a needed. Does this module do that? Please guide me through.
Thanks Prince
The feed importer has a setting that specifies the time interval between automatic imports. On the feed admin screen, find the "Basic settings" > "Refresh" option.
A really dumb question. The tokens expire after sometime. Don't they? In that case a new token will have to be created which will need my presence. No?
That's right, you need to re-login.
What if I store my facebook username and password in db? Will it be possible to auto login?
Not that I know. The whole point of OAuth is that you only use your username/password in an interactive setting, not programmatically.
Right! My bad :-( Thanks for bearing with me though! My requirement is such that I want to aggregate content from facebook pages. And I want to do that programatically because I cannot be around all the time. Can you guide me as to how I would do that?
The OAuth token that this module stores in the DB is all that's needed to keep the feed working in the background. Facebook might expire them after a number of days, and I haven't researched refreshing them beyond this point.
Okkay. Thanks a ton for this info.
Great post! I'm already using this on two sites.
My only hangup now is getting the Feeds Image Grabber module working. I believe I have everything setup but it doesn't seem to be working. Has anyone had any success with this?
I would just store the URL but it looks like some of the images are authenticated so I would get broken images on the frontend.
The description still works - thanks for the writeup.
The only thing that took me a while to find out was, that the "scope" within feeds_oauth should include "read_stream" if you want to read /me/home. To reset the oauth, just truncate the oauth_token table from the database.
{ "data": [ { "from": { "name": "Disaster.TV", "category": "News/media", }, "message": "Disaster TV expects Japan to raise", "comments": { "data": [ { "from": { "name": "Renjay September", }, "message": "Thanx guys,,,Bye!", }, { "from": { "name": "Ranjith Jey", }, "message": "Gd nyt", } ], } }, ], }
Hello. I have a json like this. Context $data.* contains from,message,comments. What will be the values of JSONPath expression to put in JSONPath Parser Settings for parsing $data.from.name and $data.comments.data.from.name and $data.comments.data.message as cck fields. Also is it possible to parse those two section of $data.comments.data.* as cck fields.
hi can you help me? pls. i have this problem. notice: Undefined property: stdClass::$title in /home/khawshik/public_html/sites/all/modules/feeds/feeds.module on line 345. Could not retrieve title from feed. And heres a debug msg And this is how the debug looks. context: array ( 'id' => '100001562085366_150851048310297', 'from' => array ( 'name' => 'Alif Hasan', 'id' => '100001562085366', ), 'message' => 'CINTA is VALOBASA....GooD MorninG...@ALL FriendS..', 'type' => 'status', 'application' => array ( 'name' => 'Mobile', 'id' => '186529194704290', ), 'created_time' => '2011-03-25T00:37:50+0000', 'updated_time' => '2011-03-25T00:37:50+0000', ) jsonpath_parser:0: 'natok 3................joy' jsonpath_parser:1: '100001533788745_188096421233245' jsonpath_parser:2: 'bangla eid natok 2008 december'
Make sure you have associated:
This is what's working for me. If you're using a new version of Feeds, there might be some changes because I haven't tested this recipe since November 2010.
In your attach file name facebook_feed-6.x-1.0.tar creates a content type Facebook Feed don't have title and body field. Thats creates the problem. i just replace that with a new one.
Thanks. i already fix that. in importer basic setting i Attached new content type .. and title is ok description is ok .. and its work ..
this great stuff. I have been working with a not-for-profit organization with an active facebook page and we would like to be able to show the content from the page on the org's own website. This module here looks like just the ticket. I am more of a drupal builder than a php dev and don't have much personal experience with feeds or json, but I get the general idea.
Wondering, as this post here is several months old, if you had made any advancement or had plans to turn this into an official d.o module?? I put it in my queue to explore this though.
Thanks!
Thanks for your comment. I am not planning to turn this into a module, since it just involves assembling and configuring components. I haven't worked on this since then, but please post your experience here and I will gladly provide assistance.
Thanks for a great post and sharing your code. I am almost there in making it work. This is really awesome to me.