fullstackpython.github.com/task-queues.html at 496f748de6fc909953127753caff034632501fe8 · PythonNuts/fullstackpython.github.com · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
<!DOCTYPE html>
<html lang="en"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
    <meta charset="utf-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="description" content="Full Stack Python shows how an entire Python web application is built and deployed. Each section of the guide explains a different key concept, from the server through the Python WSGI web framework to the front end JavaScript.">
    <meta name="author" content="Matt Makai">
    <link rel="shortcut icon" href="theme/img/full-stack-python-logo-bw.png">
    <title>Full Stack Python</title>
    <!-- Bootstrap core CSS -->
    <link href="theme/css/fsp.css" rel="stylesheet">
    <!--[if lt IE 9]>
      <script src="https://oss.maxcdn.com/libs/html5shiv/3.7.0/html5shiv.js"></script>
      <script src="https://oss.maxcdn.com/libs/respond.js/1.3.0/respond.min.js"></script>
    <![endif]-->
    <style>
    html,
    body {
      font-size: 18px;
      color: #222;
      background: #fefefe;
    }
    body {
      padding-top: 30px;
    }
    .footer {
      padding: 20px 0 30px 0;
    }
    a, a:hover {border-bottom: 1px dotted; color: #444;}
    a:hover {text-decoration: none; color: #000;}
   .logo-title {font-size: 56px; color: #403072; padding-top: 80px;
                font-family: "News Cycle", "Arial Narrow Bold", sans-serif;
                font-weight: bold; line-height: 30px; margin-left: 5px;}
    .logo-title a, .logo-title a:hover {color: #000; text-decoration: none;
                                        border-bottom: none;}
    .logo-title a:hover {color: gray;}
    .logo-image {vertical-align: top; border: none;}
    a.list-group-item.active {background: #444; border: 1px solid #222;}
    a.list-group-item.active:hover {background: #444; border: 1px solid #222;}
    #sidebar {margin-top: 30px;}

    @media (max-width: 600px) {
        .logo-header-section {
          margin: 20px 32px 0 0;
        }
    }
    </style>

    <script type="text/javascript">
      var _gaq = _gaq || [];
      _gaq.push(['_setAccount', 'UA-19910497-7']);
  _gaq.push(['_trackPageview']);

  (function() {
    var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
    ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
    var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
  })();
    </script>
  </head>

<body>
    <a href="https://github.com/makaimc/fullstackpython.github.com" class="github">
        <img style="position: absolute; top: 0; right: 0; border: 0;" src="http://s3.amazonaws.com/github/ribbons/forkme_right_darkblue_121621.png" alt="Fork me on GitHub" />
    </a>
    <div class="container">
        <div class="row">
    <div class="col-md-12">
        <div class="logo-header-section">
            <a href="/" style="text-decoration: none; border: none;"><img src="theme/img/full-stack-python-logo-bw.png" height="42" width="42" class="logo-image" /></a>
            <span class="logo-title"><a href="/">Full Stack Python</a></span>
        </div>
    </div>
</div>        <div class="row">
    <div class="col-md-8">
      <h1>Task Queues</h1>
      <p>Task queues handle asynchronous jobs that need to be processed outside the
usual HTTP request-response cycle. The most common types of jobs for task
queues include</p>
<ul class="simple">
<li>calculating computationally expensive data analytics</li>
<li>scheduling periodic jobs such as batch processes</li>
<li>spreading out large numbers of independent database inserts over time
instead of all at once</li>
</ul>
<div class="section" id="task-queue-resources">
<h2>Task Queue Resources</h2>
<p>The <a class="reference external" href="http://www.celeryproject.org/">Celery</a> distributed task queue is the
most commonly used Python library for handling asynchronous tasks and
scheduling.</p>
<p>The <a class="reference external" href="http://python-rq.org/">RQ (Redis Queue)</a> is a simple Python
library for queueing jobs and processing them in the background with workers.
It is backed by Redis and it is designed to have a low barrier to entry.
It should be integrated in your web stack easily.</p>
</div>

                                                                                                                                                                                                                                                    <br/>
            Next read the
            <a href="/static-content.html">static content</a> section.

                                                                                                                                                                                                                                                                                                  </div>
    <div class="col-md-offset-1 col-md-3" id="sidebar">
  <div class="list-group">
          <a href="/introduction.html" class="list-group-item ">Introduction</a>
          <a href="/servers.html" class="list-group-item ">Servers</a>
          <a href="/operating-systems.html" class="list-group-item ">Operating Systems</a>
          <a href="/web-servers.html" class="list-group-item ">Web Servers</a>
          <a href="/platform-as-a-service.html" class="list-group-item ">Platform-as-a-service</a>
          <a href="/databases.html" class="list-group-item ">Databases</a>
          <a href="/wsgi-servers.html" class="list-group-item ">WSGI Servers</a>
          <a href="/web-frameworks.html" class="list-group-item ">Web Frameworks</a>
          <a href="/task-queues.html" class="list-group-item active">Task Queues</a>
          <a href="/static-content.html" class="list-group-item ">Static Content</a>
          <a href="/caching.html" class="list-group-item ">Caching</a>
          <a href="/web-browsers.html" class="list-group-item ">Web Browsers</a>
          <a href="/web-application-security.html" class="list-group-item ">Web Application Security</a>
          <a href="/monitoring.html" class="list-group-item ">Monitoring</a>
          <a href="/web-analytics.html" class="list-group-item ">Web Analytics</a>
          <a href="/api-integration.html" class="list-group-item ">API Integration</a>
          <a href="/source-control.html" class="list-group-item ">Source Control</a>
          <a href="/configuration-management.html" class="list-group-item ">Configuration Management</a>
          <a href="/dependency-management.html" class="list-group-item ">Application Dependencies</a>
          <a href="/no-sql-datastore.html" class="list-group-item ">NoSQL Data Stores</a>
          <a href="/about-author.html" class="list-group-item ">About</a>
          <a href="/change-log.html" class="list-group-item ">Change Log</a>
      </div>
</div></div>
        <hr/>
        <div class="footer pull-right">
            <a href="http://www.mattmakai.com/" class="underline">Matt Makai</a> 2014
        </div>
    </div>

    <script src="http://code.jquery.com/jquery-2.1.0.min.js"></script>
    <script src="theme/js/bootstrap.min.js"></script>
</body>
</html>