5 ########################################## |
5 ########################################## |
6 |
6 |
7 Introduction |
7 Introduction |
8 ------------ |
8 ------------ |
9 |
9 |
10 tweetstream provides a class, TweetStream, that can be used to get |
10 tweetstream provides two classes, SampleStream and FollowStream, that can be |
11 tweets from Twitter's streaming API. An instance of the class can be used as |
11 used to get tweets from Twitter's streaming API. An instance of one of the |
12 an iterator. In addition to fetching tweets, the object keeps track of |
12 classes can be used as an iterator. In addition to fetching tweets, the |
13 the number of tweets collected and the rate at which tweets are received. |
13 object keeps track of the number of tweets collected and the rate at which |
|
14 tweets are received. |
14 |
15 |
15 Subclasses are available for accessing the "track" and "follow" streams |
16 SampleStream delivers a sample of all tweets. FilterStream delivers |
16 as well. |
17 tweets that match one or more criteria. Note that it's not possible |
17 |
18 to get all tweets without access to the "firehose" stream, which |
18 There's also a ReconnectingTweetStream class that handles automatic |
19 is not currently avaliable to the public. |
19 reconnecting. |
|
20 |
20 |
21 Twitter's documentation about the streaming API can be found here: |
21 Twitter's documentation about the streaming API can be found here: |
22 http://apiwiki.twitter.com/Streaming-API-Documentation . |
22 http://dev.twitter.com/pages/streaming_api_methods . |
23 |
23 |
24 **Note** that the API is blocking. If for some reason data is not immediatly |
24 **Note** that the API is blocking. If for some reason data is not immediatly |
25 available, calls will block until enough data is available to yield a tweet. |
25 available, calls will block until enough data is available to yield a tweet. |
26 |
26 |
27 Examples |
27 Examples |
28 -------- |
28 -------- |
29 |
29 |
30 Printing all incomming tweets: |
30 Printing incoming tweets: |
31 |
31 |
32 >>> stream = tweetstream.TweetStream("username", "password") |
32 >>> stream = tweetstream.SampleStream("username", "password") |
33 >>> for tweet in stream: |
33 >>> for tweet in stream: |
34 ... print tweet |
34 ... print tweet |
35 |
35 |
36 |
36 |
37 The stream object can also be used as a context, as in this example that |
37 The stream object can also be used as a context, as in this example that |
38 prints the author for each tweet as well as the tweet count and rate: |
38 prints the author for each tweet as well as the tweet count and rate: |
39 |
39 |
40 >>> with tweetstream.TweetStream("username", "password") as stream |
40 >>> with tweetstream.SampleStream("username", "password") as stream |
41 ... for tweet in stream: |
41 ... for tweet in stream: |
42 ... print "Got tweet from %-16s\t( tweet %d, rate %.1f tweets/sec)" % ( |
42 ... print "Got tweet from %-16s\t( tweet %d, rate %.1f tweets/sec)" % ( |
43 ... tweet["user"]["screen_name"], stream.count, stream.rate ) |
43 ... tweet["user"]["screen_name"], stream.count, stream.rate ) |
44 |
44 |
45 |
45 |
51 ... print "Got tweet from %-16s\t( tweet %d, rate %.1f tweets/sec)" % ( |
51 ... print "Got tweet from %-16s\t( tweet %d, rate %.1f tweets/sec)" % ( |
52 ... tweet["user"]["screen_name"], stream.count, stream.rate ) |
52 ... tweet["user"]["screen_name"], stream.count, stream.rate ) |
53 ... except tweetstream.ConnectionError, e: |
53 ... except tweetstream.ConnectionError, e: |
54 ... print "Disconnected from twitter. Reason:", e.reason |
54 ... print "Disconnected from twitter. Reason:", e.reason |
55 |
55 |
56 To get tweets that relate to specific terms, use the TrackStream: |
56 To get tweets that match specific criteria, use the FilterStream. FilterStreams |
|
57 take three keyword arguments: "locations", "follow" and "track". |
|
58 |
|
59 Locations are a list of bounding boxes in which geotagged tweets should originate. |
|
60 The argument should be an iterable of longitude/latitude pairs. |
|
61 |
|
62 Track specifies keywords to track. The argument should be an iterable of |
|
63 strings. |
|
64 |
|
65 Follow returns statuses that reference given users. Argument should be an iterable |
|
66 of twitter user IDs. The IDs are userid ints, not the screen names. |
57 |
67 |
58 >>> words = ["opera", "firefox", "safari"] |
68 >>> words = ["opera", "firefox", "safari"] |
59 >>> with tweetstream.TrackStream("username", "password", words) as stream |
69 >>> people = [123,124,125] |
|
70 >>> locations = ["-122.75,36.8", "-121.75,37.8"] |
|
71 >>> with tweetstream.FilterStream("username", "password", track=words, |
|
72 ... follow=people, locations=locations) as stream |
60 ... for tweet in stream: |
73 ... for tweet in stream: |
61 ... print "Got interesting tweet:", tweet |
74 ... print "Got interesting tweet:", tweet |
62 |
75 |
63 To get only tweets from a set of users, use the FollowStream. The following |
|
64 would get tweets for user 1, 42 and 8675309 |
|
65 |
76 |
66 >>> users = [1, 42, 8675309] |
77 Deprecated classes |
67 >>> with tweetstream.FollowStream("username", "password", users) as stream |
78 ------------------ |
68 ... for tweet in stream: |
|
69 ... print "Got tweet from:", tweet["user"]["screen_name"] |
|
70 |
79 |
71 |
80 tweetstream used to contain the classes TweetStream, FollowStream, TrackStream |
72 Simple tweet fetcher that sends tweets to an AMQP message server using carrot: |
81 LocationStream and ReconnectingTweetStream. These were deprecated when twitter |
73 |
82 changed its API end points. The same functionality is now available in |
74 >>> from carrot.messaging import Publisher |
83 SampleStream and FilterStream. The deprecated methods will emit a warning when |
75 >>> from carrot.connection import AMQPConnection |
84 used, but will remain functional for a while longer. |
76 >>> from tweetstream import TweetStream |
|
77 >>> amqpconn = AMQPConnection(hostname="localhost", port=5672, |
|
78 ... userid="test", password="test", |
|
79 ... vhost="test") |
|
80 >>> publisher = Publisher(connection=amqpconn, |
|
81 ... exchange="tweets", routing_key="stream") |
|
82 >>> with TweetStream("username", "password") as stream: |
|
83 ... for tweet in stream: |
|
84 ... publisher.send(tweet) |
|
85 >>> publisher.close() |
|
86 |
85 |
87 |
86 |
88 Changelog |
87 Changelog |
89 --------- |
88 --------- |
90 |
89 |