| author | Samuel Huron <samuel.huron@centrepompidou.fr> |
| Tue, 26 Apr 2011 10:53:35 +0200 | |
| changeset 121 | 2b794b7901d6 |
| parent 12 | 4daf47fcf792 |
| child 13 | 79b6e132e3d7 |
| permissions | -rw-r--r-- |
|
12
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
1 |
.. -*- restructuredtext -*- |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
2 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
3 |
########################################## |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
4 |
tweetstream - Simple twitter streaming API |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
5 |
########################################## |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
6 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
7 |
Introduction |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
8 |
------------ |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
9 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
10 |
tweetstream provides a class, TweetStream, that can be used to get |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
11 |
tweets from Twitter's streaming API. An instance of the class can be used as |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
12 |
an iterator. In addition to fetching tweets, the object keeps track of |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
13 |
the number of tweets collected and the rate at which tweets are received. |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
14 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
15 |
Subclasses are available for accessing the "track" and "follow" streams |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
16 |
as well. |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
17 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
18 |
There's also a ReconnectingTweetStream class that handles automatic |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
19 |
reconnecting. |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
20 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
21 |
Twitter's documentation about the streaming API can be found here: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
22 |
http://apiwiki.twitter.com/Streaming-API-Documentation . |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
23 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
24 |
**Note** that the API is blocking. If for some reason data is not immediatly |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
25 |
available, calls will block until enough data is available to yield a tweet. |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
26 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
27 |
Examples |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
28 |
-------- |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
29 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
30 |
Printing all incomming tweets: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
31 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
32 |
>>> stream = tweetstream.TweetStream("username", "password")
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
33 |
>>> for tweet in stream: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
34 |
... print tweet |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
35 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
36 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
37 |
The stream object can also be used as a context, as in this example that |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
38 |
prints the author for each tweet as well as the tweet count and rate: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
39 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
40 |
>>> with tweetstream.TweetStream("username", "password") as stream
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
41 |
... for tweet in stream: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
42 |
... print "Got tweet from %-16s\t( tweet %d, rate %.1f tweets/sec)" % ( |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
43 |
... tweet["user"]["screen_name"], stream.count, stream.rate ) |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
44 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
45 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
46 |
Stream objects can raise ConnectionError or AuthenticationError exceptions: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
47 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
48 |
>>> try: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
49 |
... with tweetstream.TweetStream("username", "password") as stream
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
50 |
... for tweet in stream: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
51 |
... print "Got tweet from %-16s\t( tweet %d, rate %.1f tweets/sec)" % ( |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
52 |
... tweet["user"]["screen_name"], stream.count, stream.rate ) |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
53 |
... except tweetstream.ConnectionError, e: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
54 |
... print "Disconnected from twitter. Reason:", e.reason |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
55 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
56 |
To get tweets that relate to specific terms, use the TrackStream: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
57 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
58 |
>>> words = ["opera", "firefox", "safari"] |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
59 |
>>> with tweetstream.TrackStream("username", "password", words) as stream
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
60 |
... for tweet in stream: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
61 |
... print "Got interesting tweet:", tweet |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
62 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
63 |
To get only tweets from a set of users, use the FollowStream. The following |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
64 |
would get tweets for user 1, 42 and 8675309 |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
65 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
66 |
>>> users = [1, 42, 8675309] |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
67 |
>>> with tweetstream.FollowStream("username", "password", users) as stream
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
68 |
... for tweet in stream: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
69 |
... print "Got tweet from:", tweet["user"]["screen_name"] |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
70 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
71 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
72 |
Simple tweet fetcher that sends tweets to an AMQP message server using carrot: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
73 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
74 |
>>> from carrot.messaging import Publisher |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
75 |
>>> from carrot.connection import AMQPConnection |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
76 |
>>> from tweetstream import TweetStream |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
77 |
>>> amqpconn = AMQPConnection(hostname="localhost", port=5672, |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
78 |
... userid="test", password="test", |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
79 |
... vhost="test") |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
80 |
>>> publisher = Publisher(connection=amqpconn, |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
81 |
... exchange="tweets", routing_key="stream") |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
82 |
>>> with TweetStream("username", "password") as stream:
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
83 |
... for tweet in stream: |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
84 |
... publisher.send(tweet) |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
85 |
>>> publisher.close() |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
86 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
87 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
88 |
Changelog |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
89 |
--------- |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
90 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
91 |
See the CHANGELOG file |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
92 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
93 |
Contact |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
94 |
------- |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
95 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
96 |
The author is Rune Halvorsen <runefh@gmail.com>. The project resides at |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
97 |
http://bitbucket.org/runeh/tweetstream . If you find bugs, or have feature |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
98 |
requests, please report them in the project site issue tracker. Patches are |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
99 |
also very welcome. |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
100 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
101 |
License |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
102 |
------- |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
103 |
|
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
104 |
This software is licensed under the ``New BSD License``. See the ``LICENCE`` |
|
4daf47fcf792
move lib and prepare tweetstream for modification
Yves-Marie Haussonne <1218002+ymph@users.noreply.github.com>
parents:
diff
changeset
|
105 |
file in the top distribution directory for the full license text. |