diff --git a/content/pages/01-introduction/01-learning-programming.markdown b/content/pages/01-introduction/01-learning-programming.markdown
index 217546782..08f676d8e 100644
--- a/content/pages/01-introduction/01-learning-programming.markdown
+++ b/content/pages/01-introduction/01-learning-programming.markdown
@@ -125,11 +125,6 @@ repositories and sites with practice problems and solutions:
* [TeachCraft](https://teachcraft.net/) combines Minecraft with Python to
learn coding.
-* [500 Data Structures and Algorithms practice problems and their solutions](https://techiedelight.quora.com/500-Data-Structures-and-Algorithms-practice-problems-and-their-solutions)
- covers a large swath of the computer science space. It is not important
- to know all of these algorithms and data structures but experience with
- many of them will be greatly beneficial in becoming a better developer.
-
### First-hand advice
These articles are written by programmers who explain how they learned to
@@ -145,11 +140,6 @@ give example paths you can think about taking as a beginner:
including persistence, respecting others and considering ideas that are
outside your comfort zone.
-* [Mastering programming](https://www.facebook.com/notes/kent-beck/mastering-programming/1184427814923414)
- by [Kent Beck](https://en.wikipedia.org/wiki/Kent_Beck) contains
- patterns and observations for how experienced programmers he has worked
- with in the past became great software developers.
-
* [This Picture Will Change the Way You Learn to Code](https://dev.to/nextdotxyz/this-picture-will-change-the-way-you-learn-tocode-4kmh)
covers a well done graphics of many up-to-date concepts and tools that
developers use. The post reminds you that you will not and should not learn
@@ -170,10 +160,6 @@ your teaching experience:
is an awesome resource that explains how you can use simple but fun
drawings to teach otherwise difficult technical concepts to students.
-* [Teaching programming to working professionals](http://pgbovine.net/PG-Podcast-21-Trey-Hunner.htm)
- is a video podcast with [Trey Hunter](https://twitter.com/treyhunner)
- about his experience teaching Python to experienced professionals.
-
* [Teach Yourself Computer Science](https://teachyourselfcs.com/) is
intended as a self-teaching tool with many resources that are classic
computer science textbooks. There are also nice explanations for why
diff --git a/content/pages/01-introduction/02-python-programming-language.markdown b/content/pages/01-introduction/02-python-programming-language.markdown
index 87a44028b..a3b046bb1 100644
--- a/content/pages/01-introduction/02-python-programming-language.markdown
+++ b/content/pages/01-introduction/02-python-programming-language.markdown
@@ -79,9 +79,6 @@ Dictionary comprehension:
### General Python language resources
-* The [online Python tutor](http://www.pythontutor.com/) visually walks
- through code and shows how it executes on the Python interpreter.
-
* [Python Module of the Week](http://pymotw.com/2/index.html) is a tour
through the Python standard library.
@@ -108,9 +105,6 @@ Dictionary comprehension:
* Armin Roacher presented [things you didn't know about Python](https://speakerdeck.com/mitsuhiko/didntknow)
at PyCon South Africa in 2012.
-* [Writing idiomatic Python](http://www.jeffknupp.com/blog/2012/10/04/writing-idiomatic-python/)
- is a guide for writing Pythonic code.
-
### Python ecosystem resources
There's an entire page on [best Python resources](/best-python-resources.html)
@@ -167,10 +161,6 @@ the very beginner topics.
covers what the code for list comprehensions looks like and gives some
example code to show how they work.
-* [An Introduction to Python Lists](http://effbot.org/zone/python-list.htm)
- is a solid overview of Python lists in general and tangentially covers
- list comprehensions.
-
### Python generator resources
* This blog post entitled
diff --git a/content/pages/01-introduction/03-why-use-python.markdown b/content/pages/01-introduction/03-why-use-python.markdown
index 505585cf3..86a0e3ecd 100644
--- a/content/pages/01-introduction/03-why-use-python.markdown
+++ b/content/pages/01-introduction/03-why-use-python.markdown
@@ -33,10 +33,7 @@ The IEEE ranked Python as the
[#1 programming language in 2019](https://spectrum.ieee.org/computing/software/the-top-programming-languages-2019),
which continued its hot streak
after ranking it
-[#1 in 2018](https://spectrum.ieee.org/at-work/innovation/the-2018-top-programming-languages),
-[#1 in 2017](https://spectrum.ieee.org/at-work/innovation/the-2017-top-programming-languages)
-and
-[#3 top programming language in 2016](http://spectrum.ieee.org/computing/software/the-2016-top-programming-languages).
+[#1 in 2018](https://spectrum.ieee.org/at-work/innovation/the-2018-top-programming-languages) and 2017.
[RedMonk's June 2019 ranking](https://redmonk.com/sogrady/2019/07/18/language-rankings-6-19/)
had Python at #3, which held consistent from previous years' rankings in
[2018](https://redmonk.com/sogrady/2018/08/10/language-rankings-6-18/)
@@ -133,13 +130,6 @@ language.
system versus statically typed languages, be sure to
[read this thorough explanation of the topic](http://blogs.perl.org/users/ovid/2010/08/what-to-know-before-debating-type-systems.html).
-* [Why I swapped C#.NET for Python as my default language and platform (and won’t be going back)](https://medium.com/@anthonypjshaw/why-i-swapped-c-net-for-python-as-my-default-language-and-platform-and-wont-be-going-back-e0063a25e491)
- provides a viewpoint from someone who is not a professional developer but
- uses coding to hack out some projects. He found Microsoft's .NET ecosystem
- lacking when it came to satisfying his needs and Python filled the gap for
- him with its wide array of open source code libraries, package management
- and ability to work well on platforms other than Windows.
-
* [Python, Machine Learning, and Language Wars](http://sebastianraschka.com/blog/2015/why-python.html)
compares Python with R, MATLAB and Julia for data science work. While
Python is great for [deployment automation](/deployment.html) and
diff --git a/content/pages/01-introduction/04-python-2-or-3.markdown b/content/pages/01-introduction/04-python-2-or-3.markdown
index eaf166982..a362e7b49 100644
--- a/content/pages/01-introduction/04-python-2-or-3.markdown
+++ b/content/pages/01-introduction/04-python-2-or-3.markdown
@@ -78,7 +78,7 @@ gone through the process and have advice for making it less painful.
implementations. There is also a
[quick reference for writting code with Python 2 and 3 compatibility](https://wiki.python.org/moin/PortingToPy3k/BilingualQuickRef).
-* [Upgrading to Python 3 with Zero Downtime](https://tech.yplanapp.com/2016/08/24/upgrading-to-python-3-with-zero-downtime/)
+* [Upgrading to Python 3 with Zero Downtime](https://adamj.eu/tech/2016/08/24/upgrading-yplan-to-python-3-with-zero-downtime/)
supplies advice on transitioning a large existing Python 2 web application
to Python 3. Their process involved upgrading dependencies, testing and
deploying the new version before going back to clean up unnecessary code
diff --git a/content/pages/01-introduction/05-enterprise-python.markdown b/content/pages/01-introduction/05-enterprise-python.markdown
index d00e8fbe0..c50052847 100644
--- a/content/pages/01-introduction/05-enterprise-python.markdown
+++ b/content/pages/01-introduction/05-enterprise-python.markdown
@@ -39,12 +39,14 @@ frameworks when otherwise they should not make technical design decisions.
## Why are there misconceptions about Python in enterprise environments?
Traditionally large organizations building enterprise software have used
-statically typed languages such as C++, .NET and Java. Throughout the 1980s
-and 1990s large companies such as Microsoft, Sun Microsystems and Oracle
-marketed these languages as "enterprise grade". The inherent snub to other
-languages was that they were not appropriate for CIOs' difficult technical
-environments. Languages other than Java, C++ and .NET were seen as risky and
-therefore not worthy of investment.
+statically typed languages and platforms such as C++, C# and Java.
+Throughout the 1990s and early 2000s, large companies such as
+Microsoft, Sun Microsystems and Oracle marketed these languages as
+"enterprise grade". The inherent message about other programming
+ecosystem was that they were not appropriate for CIOs' difficult
+technical environments. Languages other than Java, C++ and C# (along
+with its broader .NET platform) were seen as risky and therefore not
+worthy of investment.
In addition, "scripting languages" such as Python, Perl and Ruby were not
yet robust enough in the 1990s because their core standard libraries were
@@ -68,7 +70,7 @@ best maintained and fully featured pieces of code for any language.
Meanwhile, some of the traditional enterprise software development languages
such as Java have languished due to underinvestment by their major corporate
-backers. When [Oracle purchased Sun Microsystems in 2009](http://www.oracle.com/us/corporate/press/018363)
+backers. When Oracle purchased Sun Microsystems in 2009,
there was a long lag time before Java was enhanced with new language features
in Java 7. Oracle also
[bundles unwanted adware with the Java installation](http://www.engadget.com/2015/03/06/java-adware-mac/),
diff --git a/content/pages/01-introduction/06-community.markdown b/content/pages/01-introduction/06-community.markdown
index 1baec5f4f..a5f463319 100644
--- a/content/pages/01-introduction/06-community.markdown
+++ b/content/pages/01-introduction/06-community.markdown
@@ -77,7 +77,7 @@ resources provide perspective on offline events like
and
[PyCon US 2016](http://www.dreisbach.us/blog/pycon-2016/). There are
many other retrospectives for other
- [community-led conferences such as EuroPython](http://www.artima.com/weblogs/viewpost.jsp?thread=261930).
+ [community-led conferences such as EuroPython](https://www.artima.com/weblogs/viewpost.jsp?thread=261930).
These summaries can be a great way to get a slice of the experience
before purchasing a ticket and booking a trip.
@@ -112,16 +112,11 @@ the community.
provides a starter page with links to community-run newsletters, resources
and conferences.
-* There are many large active online communities on
- [Reddit](https://www.reddit.com/r/python) and
- [IRC channels](https://freenode.net/) such as #python, #python-dev
- and #distutils.
-
* The Python community has a concept known as "Benevolent Dictator For Life"
that may appear odd to newcomers. Essentially, Guido Van Rossum created
the language and still has the ability to decide community arguments one
way or the other. This post on the
- [origin of BDFL](http://www.artima.com/weblogs/viewpost.jsp?thread=235725)
+ [origin of BDFL](https://www.artima.com/weblogs/viewpost.jsp?thread=235725)
has more context about Guido's role.
* [Python Community and Python at Dropbox](https://talkpython.fm/episodes/show/30/python-community-and-python-at-dropbox)
diff --git a/content/pages/01-introduction/07-companies.markdown b/content/pages/01-introduction/07-companies.markdown
index 7eb7a78f2..c4e02cfc8 100644
--- a/content/pages/01-introduction/07-companies.markdown
+++ b/content/pages/01-introduction/07-companies.markdown
@@ -37,7 +37,7 @@ below).
at job descriptions on sites like
[Glassdoor with "Python Goldman Sachs" keywords](https://www.glassdoor.com/Jobs/Goldman-Sachs-python-Jobs-EI_IE2800.0,13_KO14,20.htm)
and
- [Indeed for JP Morgan Chase](https://www.indeed.com/salaries/Python-Developer-Salaries-at-JPMorgan-Chase).
+ [Indeed for JP Morgan Chase](https://www.indeed.com/cmp/JPMorgan-Chase/salaries/Python-Developer).
Salaries and responsibilities vary widely based on the role and whether
Python is used for data analysis,
[web application development](/web-development.html) or DevOps.
diff --git a/content/pages/01-introduction/08-best-python-resources.markdown b/content/pages/01-introduction/08-best-python-resources.markdown
index 81113487a..6677ba3d2 100644
--- a/content/pages/01-introduction/08-best-python-resources.markdown
+++ b/content/pages/01-introduction/08-best-python-resources.markdown
@@ -47,13 +47,6 @@ should skip down to the next section for "experienced developers".
and problems rather than jumping into a specific language that's recommended
to you by a friend.
-* [A Python Crash Course](https://www.grahamwheeler.com/posts/python-crash-course.html)
- gives an awesome overview of the history of Python, what drives the
- programming community and dives into example code. You will likely need
- to read this in combination with other resources to really let the syntax
- sink in, but it's a great article to read several times over as you
- continue to learn.
-
* The [Python projects tag](https://www.twilio.com/blog/tag/python) on the
Twilio blog is constantly updated with fun tutorials you can build to
learn Python, such as the
@@ -61,9 +54,6 @@ should skip down to the next section for "experienced developers".
[Choose Your Own Adventures Presentations using Flask and WebSockets](https://www.twilio.com/blog/2014/11/choose-your-own-adventure-presentations-with-reveal-js-python-and-websockets.html)
and [Martianify Photos with OpenCV](https://www.twilio.com/blog/2015/11/getting-started-with-opencv-and-python-featuring-the-martian-2.html).
-* [A Byte of Python](http://www.swaroopch.com/notes/python/) is a beginner's
- tutorial for the Python language.
-
* Google put together a great compilation of materials and subjects you
should read and learn from if you want to be a
[professional programmer](https://www.google.com/about/careers/students/guide-to-technical-development.html).
diff --git a/content/pages/01-introduction/09-best-python-videos.markdown b/content/pages/01-introduction/09-best-python-videos.markdown
index 85000438e..14be9bccd 100644
--- a/content/pages/01-introduction/09-best-python-videos.markdown
+++ b/content/pages/01-introduction/09-best-python-videos.markdown
@@ -163,7 +163,7 @@ like [PyCon US](https://us.pycon.org/) and
Python language.
* PyCon US videos from
- [2020](https://www.youtube.com/c/PyCon2020/videos),
+ [2020](https://www.youtube.com/playlist?list=PL2Uw4_HvXqvbpFquYIE57BEAqkQWk-iFg),
[2019](https://www.youtube.com/channel/UCxs2IIVXaEHHA4BtTiWZ2mQ/videos),
[2018](https://www.youtube.com/channel/UCsX05-2sVSH7Nx3zuk3NYuQ/videos),
[2017](https://www.youtube.com/channel/UCrJhliKNQ8g0qoE_zvL8eVg/videos),
diff --git a/content/pages/01-introduction/10-best-python-podcasts.markdown b/content/pages/01-introduction/10-best-python-podcasts.markdown
index f0181a269..a2c528fee 100644
--- a/content/pages/01-introduction/10-best-python-podcasts.markdown
+++ b/content/pages/01-introduction/10-best-python-podcasts.markdown
@@ -42,16 +42,16 @@ listen and learn.
[web application development](/web-development.html) in Python using the
[Django web framework](/django.html).
+* [The Real Python Podcast](https://realpython.com/podcasts/rpp/) is a weekly
+ podcast with interviews, coding tips, and conversation with guests from the
+ Python community.
+
* [Teaching Python](https://www.teachingpython.fm/) is a podcast by two
teachers about their adventures teaching middle school computer science,
problem solving, handling failure, frustration, and success with
teaching Python programming.
-* Professor Philip Guo has a video podcast called
- [PG Podcast](https://podcasts.apple.com/us/podcast/philip-guo-podcasts-pgbovine-net/id1276072242),
- which typically covers Python subjects.
-* [The Real Python Podcast](https://realpython.com/podcasts/rpp/) is a weekly podcast with interviews, coding tips, and conversation with guests from the Python community.
## Favorite podcast episodes
Here are a list of my favorite episodes from various Python podcasts before
diff --git a/content/pages/02-development-environments/00-development-environments.markdown b/content/pages/02-development-environments/00-development-environments.markdown
index cb27c7ee4..4b7dc8b94 100644
--- a/content/pages/02-development-environments/00-development-environments.markdown
+++ b/content/pages/02-development-environments/00-development-environments.markdown
@@ -121,12 +121,6 @@ configuration as a starting point and customize it from there.
[which version of Python to use](/python-2-or-3.html) and adding
[code metrics](/code-metrics.html) libraries for checking syntax.
-* [Three Ways to Install Python on your Windows Computer](http://blog.yhat.com/posts/installing-python-on-windows.html)
- provides multiple avenues for Windows users to get Python on their machine
- before setting up the rest of their development environment. Unlike
- macOS and Linux, the Windows [operating system](/operating-systems.html)
- does not include Python with its default installation.
-
* [PyCharm: The Good Parts](http://nafiulis.me/pycharm-the-good-parts-i.html)
shows you how to be more efficient and productive with that IDE if it's
your choice for writing Python code.
diff --git a/content/pages/02-development-environments/02-vim.markdown b/content/pages/02-development-environments/02-vim.markdown
index da89bbe3d..f37c04a5d 100644
--- a/content/pages/02-development-environments/02-vim.markdown
+++ b/content/pages/02-development-environments/02-vim.markdown
@@ -91,6 +91,31 @@ If a Vimrc file does not already exist, just create it within the user's
home directory and it will be picked up by Vim the next time you open the
editor.
+The following are a few resources for learning what to put in, and how to
+structure a `.vimrc` file. I recommend adding configuration options one
+at a time to test them individually instead of going whole hog with a Vimrc
+you are unfamiliar with.
+
+* [A Good Vimrc](http://dougblack.io/words/a-good-vimrc.html) is a fantastic,
+ detailed overview and opinionated guide to configuring Vim. Highly
+ recommended for new and experienced Vim users.
+
+* [5 lines for a blank .vimrc](https://swordandsignals.com/2020/12/13/5-lines-in-vimrc.html)
+ shows settings for case insensitive search, highlighting as you search,
+ disabling swap, and a couple more "must have" enhancements to the
+ default configuration.
+
+* [Vim and Python](https://justin.abrah.ms/vim/vim_and_python.html) shows
+ and explains many Python-specific .vimrc options.
+
+* This
+ [repository's folder with Vimrc files](https://github.com/amix/vimrc/tree/master/vimrcs)
+ has example configurations that are well commented and easy to learn from.
+
+* For people who are having trouble getting started with Vim, check out this
+ blog post on the
+ [two simple steps that helped this author learn Vim](http://adamdelong.com/two-simple-steps-helped-me-learn-vim/).
+
### Vim tutorials
Vim has a reputation for a difficult learning curve, but it's much easier
@@ -142,11 +167,6 @@ to get started with these tutorials.
* [Vim Adventures](http://vim-adventures.com/) is a cute, fun browser-based
game that helps you learn Vim commands by playing through the adventure.
-* In [Vim: revisited](http://mislav.uniqpath.com/2011/12/vim-revisited/) the
- author explains his on-again off-again relationship with using Vim. He then
- shows how he configures and uses the editor so it sticks as his primary
- code editing tool.
-
* [Things About Vim I Wish I Knew Earlier](https://blog.petrzemek.net/2016/04/06/things-about-vim-i-wish-i-knew-earlier/)
explores the lessons one developer learned while exclusively using Vim
for several years. The author includes using relative instead of absolute
@@ -159,31 +179,6 @@ to get started with these tutorials.
habits.
-### Vimrc resources
-These are a few resources for learning how to structure a `.vimrc` file. I
-recommend adding configuration options one at a time to test them
-individually instead of going whole hog with a Vimrc you are unfamiliar with.
-
-* [A Good Vimrc](http://dougblack.io/words/a-good-vimrc.html) is a fantastic,
- detailed overview and opinionated guide to configuring Vim. Highly
- recommended for new and experienced Vim users.
-
-* [Vim and Python](https://justin.abrah.ms/vim/vim_and_python.html) shows
- and explains many Python-specific .vimrc options.
-
-* [Vim as a Python IDE](http://liuchengxu.org/posts/use-vim-as-a-python-ide/)
- shows a slew of plugins and configuration options for coding with Python
- in Vim.
-
-* This
- [repository's folder with Vimrc files](https://github.com/amix/vimrc/tree/master/vimrcs)
- has example configurations that are well commented and easy to learn from.
-
-* For people who are having trouble getting started with Vim, check out this
- blog post on the
- [two simple steps that helped this author learn Vim](http://adamdelong.com/two-simple-steps-helped-me-learn-vim/).
-
-
### Vim installation guides
These installation guides will help you get Vim up and running on Mac OS X,
Linux and Windows.
diff --git a/content/pages/02-development-environments/04-sublime-text.markdown b/content/pages/02-development-environments/04-sublime-text.markdown
index 841687737..196df3b51 100644
--- a/content/pages/02-development-environments/04-sublime-text.markdown
+++ b/content/pages/02-development-environments/04-sublime-text.markdown
@@ -68,10 +68,6 @@ links should get your editor customized with linters,
is a spectacular tutorial that covers installing Sublime Text and
configuring a multitude of helpful Python programming plugins.
-* [Sublime Text 3 Heaven](https://www.kennethreitz.org/essays/sublime-text-3-heaven)
- is a quick overview of the extensions, packages and bonus toys that
- one developer uses for his own Sublime Text development setup.
-
* [Sublime Tutor](https://sublimetutor.com/) is an interactive in-editor
keyboard shortcuts tutorial that plugs into Sublime so you can learn and
become more productive as you use the editor.
diff --git a/content/pages/02-development-environments/05-pycharm.markdown b/content/pages/02-development-environments/05-pycharm.markdown
index c6810f26c..c464d6e87 100644
--- a/content/pages/02-development-environments/05-pycharm.markdown
+++ b/content/pages/02-development-environments/05-pycharm.markdown
@@ -29,10 +29,6 @@ Python code.
is a solid discussion thread with different developers' perspectives
on using PyCharm for coding their applications.
-* [How to Get Started with PyCharm and Have a Productive Python IDE](https://pedrokroger.net/getting-started-pycharm-python-ide/)
- covers the basics of configuring PyCharm for running code within
- virtualenvs, macros, using the console and code completion.
-
* [Using PyCharm with Pyramid](https://docs.pylonsproject.org/projects/pyramid-cookbook/en/latest/development_tools/pycharm.html)
is specific to developing and debugging
[with the Pyramid web framework](/pyramid.html).
diff --git a/content/pages/02-development-environments/06-jupyter-notebook.markdown b/content/pages/02-development-environments/06-jupyter-notebook.markdown
index 5b2b44a06..58c9c32b6 100644
--- a/content/pages/02-development-environments/06-jupyter-notebook.markdown
+++ b/content/pages/02-development-environments/06-jupyter-notebook.markdown
@@ -146,6 +146,13 @@ like advanced interactive visualizations.
code that is suitable for [deployment](/deployment.html) to a production
environment.
+* [How to use ipywidgets to make your Jupyter notebook interactive](https://www.wrighters.io/use-ipywidgets-with-jupyter-notebooks/)
+ is a tutorial on how to use the
+ [ipywidgets](https://ipywidgets.readthedocs.io/en/latest/) library
+ to make Jupyter Notebooks respond to users' input and go beyond
+ simply presenting data into having users be able to do some additional
+ analysis themselves.
+
* [28 Jupyter Notebook tips, tricks and shortcuts](https://www.dataquest.io/blog/jupyter-notebook-tips-tricks-shortcuts/)
explains many of the lesser-known keyboard shortcuts and mechanisms
to output settings.
diff --git a/content/pages/02-development-environments/08-bash-shell.markdown b/content/pages/02-development-environments/08-bash-shell.markdown
index c7e94db46..22c7877bc 100644
--- a/content/pages/02-development-environments/08-bash-shell.markdown
+++ b/content/pages/02-development-environments/08-bash-shell.markdown
@@ -56,12 +56,114 @@ really just what works for your brain as a developer that will guide
how often you interact with the Bash shell.
-### Bash resources
+### Getting started with Bash
+Working with a shell, Bash or otherwise, is intimidating the first time
+you try to get started. You are staring at the `$` prompt without a
+whole lot of direction.
+
+When you are completely new to using Bash, it is a good idea to at least
+scan, if not take some additional time for in-depth reading of the
+documentation for commands that every developer uses. The following
+commands are used so frequently in Bash that an experienced developer
+probably does not even think about them anymore, they become just a
+natural part of your workflow:
+
+* `echo`: [print text to the command line](https://man7.org/linux/man-pages/man1/echo.1.html)
+* `ls`: [list the contents of a directory](https://man7.org/linux/man-pages/man1/ls.1.html)
+* `cd`: [change the working directory](https://man7.org/linux/man-pages/man1/cd.1p.html)
+* `cp`: [copy a file or directory](https://man7.org/linux/man-pages/man1/cp.1.html)
+* `mv`: [move one or more files](https://man7.org/linux/man-pages/man1/mv.1.html)
+* `rm`: [delete one or more files or directories](https://man7.org/linux/man-pages/man1/rm.1.html)
+
+If you know how to use the above commands then you will at least be able
+to move around the file system, create, move and update files and know
+what is on your storage device(s).
+
+The following commands are somewhat more advanced but also frequently
+used by developers:
+
+* `su`: [run comamnds as different users or groups](https://man7.org/linux/man-pages/man1/su.1.html)
+* `whoami`: [print which user you are currently logged in as](https://man7.org/linux/man-pages/man1/whoami.1.html)
+* `grep`: [searches for patterns in files](https://man7.org/linux/man-pages/man1/grep.1.html)
+
+The above lists are not even close to exhaustive for what commands
+you need to know when working with Bash. Read some of the following
+introductory tutorials to gain a better understanding of working
+with this shell:
+
+* [The Linux command line for beginner](https://ubuntu.com/tutorials/command-line-for-beginners)
+ by [Ubuntu](/ubuntu.html) will provide you with context for how to
+ use the command line, working with files and directories, and handling
+ superuser commands.
+
* [Bash Guide for beginners](http://www.tldp.org/LDP/Bash-Beginners-Guide/html/Bash-Beginners-Guide.html)
- is an entire book for those new to working with commandlines. It covers
+ is an entire book for those new to working with command lines. It covers
commands, paths, Bash shell scripting, variables and many other critical
topics that are necessary to move from beginner to advanced Bash user.
+* [101 Bash Commands and Tips for Beginners to Experts](https://dev.to/awwsmm/101-bash-commands-and-tips-for-beginners-to-experts-30je)
+ gives a well-done laundry list of tricks to explore.
+
+* [Bash Quick References](https://shellmagic.xyz/) is a cheat sheet for
+ common operators and signals that come up when working with scripts.
+
+
+### Bash scripting
+Bash is used not only as an interactive prompt but also for scripting, which
+makes it possible to execute one or more Bash commands stored within a file.
+These scripts can be short, with only a single command, or very complicated
+with control-flow logic, for loops, and almost anything you want to automate
+or compute because
+[Bash is a Turing-complete programming language](https://www.quora.com/Is-Bash-Turing-complete).
+
+Complex Bash scripts sometimes get a negative reputation because they can be
+difficult to read and understand if you are not the original author (or you
+are reading your own script after a significant period of time has elapsed).
+There are many ways to accomplish the same tasks with Bash so the files are
+often confusing to read unless the author of a script included clear
+documentation. This readability problem is typically less of an issue with
+Python scripts because spacing is enforced and the standard library
+encapsulates common tasks.
+
+It's a good idea to think about how you want to structure your Bash scripts
+as they grow larger. The following resources provide insight into what you
+should consider while coding Bash scripts.
+
+* This [minimal safe Bash template](https://betterdev.blog/minimal-safe-bash-script-template/)
+ contains an 86-line Bash script that the author claims once you
+ understand and use it as a base then it will make your scripts
+ easier to maintain over time.
+
+* [Creating a bash completion script](https://iridakos.com/tutorials/2018/03/01/bash-programmable-completion-tutorial.html)
+ is a great tutorial that walks you through a reasonably complex Bash
+ script for completing syntax in other Bash shell scripts.
+
+* [Anybody can write good bash (with a little effort)](https://blog.yossarian.net/2020/01/23/Anybody-can-write-good-bash-with-a-little-effort)
+ covers the basics of shell scripting and provides some recommendations
+ for creating more maintainable scripts such as using linters and
+ formatters.
+
+* Google's [Shell Style Guide](https://google.github.io/styleguide/shell.xml)
+ covers how to write consistent, maintainable shell scripts, which is
+ particularly important if you have ever tried to debug a hacky shell
+ script that was never meant to be used by anyone other than the original
+ author.
+
+* [Bash scripting quirks & safety tips](https://jvns.ca/blog/2017/03/26/bash-quirks/)
+ explains Bash basic programming constructs like `for` loops and variable
+ assignment then goes into ways to avoid weird issues in your code.
+
+* If all else fails when you're trying to use Bash scripts, this article
+ on [replacing Bash scripts with Python](https://github.com/ninjaaron/replacing-bash-scripting-with-python)
+ is a guide on swapping in Python for administrative scripting, including
+ what to do about replacing invaluable command line tools such as `awk`,
+ `sed` and `grep`.
+
+
+### Additional Bash resources
+The following resources cover more advanced Bash use cases and what pitfalls
+to try to avoid as you work with the shell or write scripts.
+
* [Advancing in the Bash shell](http://samrowe.com/wordpress/advancing-in-the-bash-shell/)
covers important concepts such as bang syntax, movement commands,
tab completion and aliases.
@@ -82,19 +184,6 @@ how often you interact with the Bash shell.
`PROMPT_COMMAND`, `CDPATH` and `REPLY` which can simplify your
scripts by using values that Bash already has stored for you.
-* Google's [Shell Style Guide](https://google.github.io/styleguide/shell.xml)
- covers how to write consistent, maintainable shell scripts, which is
- particularly important if you have ever tried to debug a hacky shell
- script that was never meant to be used by anyone other than the original
- author.
-
-* [101 Bash Commands and Tips for Beginners to Experts](https://dev.to/awwsmm/101-bash-commands-and-tips-for-beginners-to-experts-30je)
- is a well-done laundry list of tricks to explore.
-
-* [Bash scripting quirks & safety tips](https://jvns.ca/blog/2017/03/26/bash-quirks/)
- explains Bash basic programming constructs like `for` loops and variable
- assignment then goes into ways to avoid weird issues in your code.
-
* [Safe ways to do things in bash](https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md)
shows you how to not shoot yourself in the foot by using safe coding
practices with your shell scripts.
@@ -112,19 +201,10 @@ how often you interact with the Bash shell.
application that generates a hostable, customizable status page for your
services.
-* [Replacing Bash scripts with Python](https://github.com/ninjaaron/replacing-bash-scripting-with-python)
- is a guide on using using Python for administrative scripting, including
- what to do about replacing invaluable command line tools such as `awk`,
- `sed` and `grep`.
-
* [Using Aliases to Speed Up Your Git Workflow](https://dev.to/robertcoopercode/using-aliases-to-speed-up-your-git-workflow-2f5a)
has a bunch of shell aliases that make it easier for you to execute
complicated or uncommon [Git](/git.html) commands.
-* [Creating a bash completion script](https://iridakos.com/tutorials/2018/03/01/bash-programmable-completion-tutorial.html)
- is a great tutorial that walks you through a reasonably complex Bash
- script for completing syntax in other Bash shell scripts.
-
* [6 Tips Before You Write Your Next Bash Cronjob](https://yasoob.me/posts/6-tips-before-you-write-your-next-bash-cronjob/)
covers starting your scripts with shebang, redirecting output, timeouts
and sudo privileges.
@@ -139,17 +219,15 @@ how often you interact with the Bash shell.
their destructive abilities by reading through the descriptions provided
by the author.
-* [Faster bash startup](https://danpker.com/posts/2020/faster-bash-startup/)
+* [Faster bash startup](https://danpker.com/posts/faster-bash-startup/)
and
[Even faster bash startup](https://work.lisk.in/2020/11/20/even-faster-bash-startup.html)
are two great tutorials that will save you a bunch of time if you frequently
open new Bash shells. On many systems you can easily cut down the startup
time for the shell which can be unnecessarily sluggish.
-* [Bash Quick References](https://shellmagic.xyz/) is a cheat sheet for
- common operators and signals that come up when working with scripts.
-
-* [Anybody can write good bash (with a little effort)](https://blog.yossarian.net/2020/01/23/Anybody-can-write-good-bash-with-a-little-effort)
- covers the basics of shell scripting and provides some recommendations
- for creating more maintainable scripts such as using linters and
- formatters.
+* [Bash HTTP monitoring dashboard](https://raymii.org/s/software/Bash_HTTP_Monitoring_Dashboard.html)
+ ([source code](https://github.com/RaymiiOrg/bash-http-monitoring))
+ is a useful application fully written in Bash shell scripts that
+ monitors the health of one or more websites to make sure they are
+ up and running.
diff --git a/content/pages/02-development-environments/12-tmux.markdown b/content/pages/02-development-environments/12-tmux.markdown
index 4da1c3016..4b95fba6c 100644
--- a/content/pages/02-development-environments/12-tmux.markdown
+++ b/content/pages/02-development-environments/12-tmux.markdown
@@ -28,14 +28,16 @@ easier to use many shells at once and attaching to both local and remote
tmux+Vim user myself, I can attest to how great these two tools complement
each other.
+* [Writing & Coding Workflow](http://jacobzelko.com/workflow/) shows one
+ developer's configuration that combines [Vim](/vim.html) and several plugins
+ with tmux for a productive setup.
+
* [Making tmux Pretty and Usable - A Guide to Customizing your tmux.conf](http://www.hamvocke.com/blog/a-guide-to-customizing-your-tmux-conf/)
* [Tmux Pairing Anywhere: On Your Box](http://iamvery.com/2013/11/16/tmux-pairing-anywhere-on-your-box.html)
* [Using tmux Properly](http://danielallendeutsch.com/blog/16-using-tmux-properly.html)
-* [Differences between tmux vs screen](https://wtanaka.com/node/8136)
-
* [The Power Of tmux Hooks](https://devel.tech/tips/n/tMuXz2lj/the-power-of-tmux-hooks/)
* There are a slew of "cheat sheets" for tmux out there, here are a few
diff --git a/content/pages/02-development-environments/14-environment-configuration.markdown b/content/pages/02-development-environments/14-environment-configuration.markdown
index 2c87e88fb..d65114eb7 100644
--- a/content/pages/02-development-environments/14-environment-configuration.markdown
+++ b/content/pages/02-development-environments/14-environment-configuration.markdown
@@ -34,6 +34,13 @@ Either one can be used in your applications but there are slight differences
that can make one better than the other in various situations.
Other useful environment variables resources:
+* [How to Set Environment Variables in Linux and Mac: The Missing Manual](https://doppler.com/blog/how-to-set-environment-variables-in-linux-and-mac)
+ is a wonderfully detailed guide with many tips and tricks throughout
+ the walkthrough such as quickly setting environment variables for a
+ single command, passing environment variables through when using sudo
+ and executing a command in a "clean" environment without everything
+ you have already set interfering or being accessible to that script.
+
* [The Twelve-Factor App](https://12factor.net/) describes a method for
securing environment data for your applications. The twelve factors are
commonly referenced across many programming ecosystems, not just Python,
diff --git a/content/pages/02-development-environments/15-application-dependencies.markdown b/content/pages/02-development-environments/15-application-dependencies.markdown
index a01f2eba9..927a3720d 100644
--- a/content/pages/02-development-environments/15-application-dependencies.markdown
+++ b/content/pages/02-development-environments/15-application-dependencies.markdown
@@ -39,9 +39,6 @@ all yourself. A few of the best collections of Python libraries are
shows the open source Python projects trending today, this week, and this
month.
-* This list of [20 Python libraries you can’t live without](http://freepythontips.wordpress.com/2013/07/30/20-python-libraries-you-cant-live-without/)
- is a wide-ranging collection from data analysis to testing tools.
-
* Wikipedia actually has an extensive
[page dedicated to Python libraries](http://en.wikipedia.org/wiki/List_of_Python_software)
grouped by categories.
@@ -161,16 +158,6 @@ so far to get up to speed on building and releasing your own packages.
provides a collection of resources to understand how to package and
distribute Python code libraries.
-* [Alice in Python projectland](https://veekaybee.github.io/2017/09/26/python-packaging/)
- is an amazing post that takes the reader from simple Python script
- into a complete Python package.
-
-* [Perils of packaging](https://malramsay.com/post/perils_of_packaging/)
- covers several edge cases that come up when trying to put together
- pieces like Travis CI, PyPI and conda. The post walks through the errors
- and how to get around them until they are smoothed out by updates to the
- tools.
-
* [How to Publish Your Package on PyPI](https://blog.jetbrains.com/pycharm/2017/05/how-to-publish-your-package-on-pypi/)
is for developers who have created a code library they would like to
share and make installable for other developers.
@@ -230,10 +217,6 @@ developers.
dependency management tools including newer ones such as pipenv and
Poetry.
-* Occasionally arguments about using Python's dependency manager versus
- one of Linux's dependency managers comes up. This provides
- [one perspective on that debate](http://notes.pault.ag/debian-python/).
-
* [Open source trust scaling](http://lucumr.pocoo.org/2016/3/24/open-source-trust-scaling/)
is a good piece for the [Python community](/python-community.html)
(and other programming communities) that is based on the
diff --git a/content/pages/02-development-environments/18-source-control.markdown b/content/pages/02-development-environments/18-source-control.markdown
index 83913183d..c96745a51 100644
--- a/content/pages/02-development-environments/18-source-control.markdown
+++ b/content/pages/02-development-environments/18-source-control.markdown
@@ -194,9 +194,9 @@ implementation.
free online version of the O'Reilly
[Version Control with Subversion](https://www.amazon.com/dp/B002SR2QIW/) book.
-* [How to use Subversion (SVN)](https://deveo.com/svn-tutorial/) lays out
- the basic concepts and provides the first few steps for getting started
- tracking files.
+* [How to Host SVN Repositories](https://www.perforce.com/blog/vcs/how-host-subversion-svn)
+ lays out the basic concepts and provides the first few steps for getting
+ started tracking files.
* [10 Most Used SVN Commands with Examples](http://www.thegeekstuff.com/2011/04/svn-command-examples/)
is a good refresher list if you've used SVN in the past but it has been
diff --git a/content/pages/02-development-environments/19-git.markdown b/content/pages/02-development-environments/19-git.markdown
index ae0fab9b8..5bffca262 100644
--- a/content/pages/02-development-environments/19-git.markdown
+++ b/content/pages/02-development-environments/19-git.markdown
@@ -174,10 +174,6 @@ workflow. These resources will come in handy for specific Git subjects.
interactive mode or on its own with `git bisect run` to find the
problematic code commit that needs to be fixed.
-* [How Microsoft uses Git](https://docs.microsoft.com/en-us/azure/devops/learn/devops-at-microsoft/use-git-microsoft)
- gives a high-level overview of their repository structure and hosting
- at the extremely large scale organization.
-
* [GitTips](https://git.wiki.kernel.org/index.php/GitTips) is a list of
pro tips to clean up common issues and how to dive through Git history
to find specific text.
@@ -244,10 +240,6 @@ minimize merge conflicts.
why at GitHub they do not use the git-flow model and provides an
alternative that solves some of the issues they found with git-flow.
-* [Git Workflows That Work](http://blog.endpoint.com/2014/05/git-workflows-that-work.html)
- is a helpful post with diagrams to show how teams can create a Git workflow
- that will help their development process.
-
* [Comparing workflows](https://www.atlassian.com/git/tutorials/comparing-workflows)
provides a slew of examples for how developers on a team can handle merge
conflicts and other situations that commonly arise when using Git.
diff --git a/content/pages/03-data/00-data.markdown b/content/pages/03-data/00-data.markdown
index a2c996e10..39a767500 100644
--- a/content/pages/03-data/00-data.markdown
+++ b/content/pages/03-data/00-data.markdown
@@ -12,7 +12,7 @@ subsections, including (in no particular order):
* data processing / wrangling
* machine learning
-* data analysis
+* [data analysis](/data-analysis.html)
* [visualization](/data-visualization.html)
* geospatial mapping
* persistence via [relational databases](/databases.html) and
@@ -154,6 +154,17 @@ sets.
have evolved over the past 20ish years based on his first-hand experience
as a leader and member in that community.
+* The State of [Python Speech Recognition](https://www.assemblyai.com/blog/the-state-of-python-speech-recognition-in-2021)
+ in 2021 is a practical overview of a specific area in data: extracting text
+ from voice recording data. Looking at verticals like this one can make it
+ easier to understand changes that are occurring in some parts of data and
+ programming that could be applied to other areas.
+
+* [Automated Data Wrangling](https://catalyst.coop/2021/05/23/automated-data-wrangling/)
+ covers cleaning, labeling, and automating the bunch of activities
+ that are typically necessary before analysis and data usage can
+ begin for a project.
+
* The [Open Source Data Science Masters](http://datasciencemasters.org/)
is a well-crafted free curriculum and set of resources for students who
want to learn both the theory and technologies for working with data.
diff --git a/content/pages/03-data/01-databases.markdown b/content/pages/03-data/01-databases.markdown
index 5eed45185..bad071a2c 100644
--- a/content/pages/03-data/01-databases.markdown
+++ b/content/pages/03-data/01-databases.markdown
@@ -100,7 +100,7 @@ Find out about Python applications with a MySQL backed on the dedicated
To work with a relational database using Python, you need to use a code
library. The most common libraries for relational databases are:
-* [psycopg2](http://initd.org/psycopg/)
+* [psycopg](https://www.psycopg.org/)
([source code](https://github.com/psycopg/psycopg2))
for PostgreSQL.
@@ -112,8 +112,6 @@ library. The most common libraries for relational databases are:
* [cx\_Oracle](https://oracle.github.io/python-cx_Oracle/index.html) for
Oracle Database ([source code](https://github.com/oracle/python-cx_Oracle)).
- Oracle moved their
- [open source driver code from SourceForge to GitHub in 2017](https://blogs.oracle.com/developers/oracle-database-python-driver-now-on-github).
SQLite support is built into Python 2.7+ and therefore a separate library
@@ -183,7 +181,7 @@ speed on SQL if you have never previously used it.
elaborates on one of the trickiest parts of writing SQL statements
that bridge one or more tables: the `JOIN`.
-* [Writing better SQL](http://www.craigkerstiens.com/2016/01/08/writing-better-sql/)
+* [Writing more legible SQL](https://www.craigkerstiens.com/2016/01/08/writing-more-legible-sql/)
is a short code styling guide to make your queries easier to read.
* [SQL Intermediate](https://www.dataquest.io/blog/sql-intermediate/) is a
@@ -265,7 +263,7 @@ speed on SQL if you have never previously used it.
1. Install PostgreSQL on your server. Assuming you went with Ubuntu run
``sudo apt-get install postgresql``.
-1. Make sure the [psycopg2](http://initd.org/psycopg/) library is in your
+1. Make sure the [psycopg](https://www.psycopg.org/) library is in your
application's dependencies.
1. Configure your web application to connect to the PostgreSQL instance.
diff --git a/content/pages/03-data/02-postgresql.markdown b/content/pages/03-data/02-postgresql.markdown
index 26954a7f8..5e700895a 100644
--- a/content/pages/03-data/02-postgresql.markdown
+++ b/content/pages/03-data/02-postgresql.markdown
@@ -57,7 +57,7 @@ architecture.
To work with relational databases in Python you need to use a database
driver, which is also referred to as a database connector. The most common
driver library for working with PostgreSQL is
-[psycopg2](http://initd.org/psycopg/). There is
+[psycopg](https://www.psycopg.org/). There is
[a list of all drivers on the PostgreSQL wiki](https://wiki.postgresql.org/wiki/Python),
including several libraries that are no longer maintained. If you're
working with the
@@ -126,13 +126,7 @@ walkthroughs I've read.
* This article explains how and why PostgreSQL can handle
[full text searching](http://blog.lostpropertyhq.com/postgres-full-text-search-is-good-enough/)
- for many use cases. If you're going down this route, read
- [this blog post that explains how one developer implemented PostgreSQL full text search with SQLAlchemy](http://blog.garage-coding.com/2015/12/18/postgres-fulltext-search.html).
-
-* [django-postgres-copy](http://django-postgres-copy.californiacivicdata.org/en/latest/)
- is a tool for bulk loading data into a PostgreSQL database based on Django models.
- [Say hello to our new open-source software for loading bulk data into PostgreSQL](http://www.californiacivicdata.org/2015/07/17/hello-django-postgres-copy/)
- is an introduction to using the tool in your own projects.
+ for many use cases.
* [How to speed up tests in Django and PostgreSQL](http://nemesisdesign.net/blog/coding/how-to-speed-up-tests-django-postgresql/)
explains some hacks for making your schema migration-backed run quicker.
diff --git a/content/pages/03-data/04-sqlite.markdown b/content/pages/03-data/04-sqlite.markdown
index 2cb4cef26..5c7f3fd00 100644
--- a/content/pages/03-data/04-sqlite.markdown
+++ b/content/pages/03-data/04-sqlite.markdown
@@ -86,6 +86,10 @@ tutorials will help you get started.
It's a great short read which shows that the code is well-tested and
maintained.
+* [SQLite is not a toy database](https://antonz.org/sqlite-is-not-a-toy-database/)
+ is a whirlwind overview of some of the best aspects of SQLite and why
+ you should use it.
+
* [Data Analysis of 8.2 Million Rows with Python and SQLite](https://plot.ly/ipython-notebooks/big-data-analytics-with-pandas-and-sqlite/)
explains how you can load a large dataset in to SQLite and visualize it
using the Plotly service.
@@ -139,6 +143,9 @@ you are having with SQLite rather than going through a general tutorial.
digs into the internals of SQLite and shows some bugs found (and
since fixed) while the author was researching the SQLite source code.
+* [How to Store Multimedia Files in a SQLite3 Database with Python](https://www.twilio.com/blog/intro-multimedia-file-upload-python-sqlite3-database)
+ goes through the Python code for storing and accessing BLOB-type objects.
+
* [Going Fast with SQLite and Python](http://charlesleifer.com/blog/going-fast-with-sqlite-and-python/)
shares essential knowledge for working effectively with SQLite in Python,
particularly when it comes to transactions, concurrency and commits.
@@ -148,11 +155,6 @@ you are having with SQLite rather than going through a general tutorial.
[object-relational mapper (ORM)](/object-relational-mappers-orms.html)
to implement virtual tables and aggregates on top of SQLite.
-* [Use SQLite with Django On AWS Lambda with Zappa](https://blog.zappa.io/posts/use-sqlite-with-django-on-aws-lambda-with-zappa)
- provides an example `dev_settings.py` file for
- locally testing a [Django](/django.html) application intended for
- [AWS Lambda](/aws-lambda.html).
-
* [SQLite Database Authorization and Access Control with Python](http://charlesleifer.com/blog/sqlite-database-authorization-and-access-control-with-python/)
covers how to control access to the SQLite database connection and
file even though SQLite normally allows unauthorized access by design.
diff --git a/content/pages/03-data/05-object-relational-mappers.markdown b/content/pages/03-data/05-object-relational-mappers.markdown
index df5ce8bb0..6451908e8 100644
--- a/content/pages/03-data/05-object-relational-mappers.markdown
+++ b/content/pages/03-data/05-object-relational-mappers.markdown
@@ -64,7 +64,7 @@ there was a pressing reason.
Python ORM libraries are not required for accessing relational
databases. In fact, the low-level access is typically provided by another
library called a *database connector*, such as
-[psycopg](http://initd.org/psycopg/) (for PostgreSQL)
+[psycopg](https://www.psycopg.org/) (for PostgreSQL)
or [MySQL-python](https://pypi.org/project/MySQL-python/1.2.5) (for
MySQL). Take a look at the table below which shows how ORMs can work with
different web frameworks and connectors and relational databases.
diff --git a/content/pages/03-data/10-no-sql-datastore.markdown b/content/pages/03-data/10-no-sql-datastore.markdown
index 05e47b705..c6c9b4683 100644
--- a/content/pages/03-data/10-no-sql-datastore.markdown
+++ b/content/pages/03-data/10-no-sql-datastore.markdown
@@ -189,10 +189,6 @@ representing a person could have a property of "female" or "male".
* [CAP Theorem overview](http://natishalom.typepad.com/nati_shaloms_blog/2010/10/nocap.html)
presents the basic constraints all databases must trade off in operation.
-* This post on [What is a NoSQL database? Learn By Writing One in Python](http://jeffknupp.com/blog/2014/09/01/what-is-a-nosql-database-learn-by-writing-one-in-python/)
- is a detailed article that breaks the mystique behind what some forms
- of NoSQL databases are doing under the covers.
-
* The [CAP Theorem series](http://blog.thislongrun.com/2015/03/the-cap-theorem-series.html)
explains concepts related to NoSQL such as what is ACID compared to CAP, CP
versus CA and high availability in large scale deployments.
diff --git a/content/pages/03-data/11-redis.markdown b/content/pages/03-data/11-redis.markdown
index c05d04741..3600ddae2 100644
--- a/content/pages/03-data/11-redis.markdown
+++ b/content/pages/03-data/11-redis.markdown
@@ -84,10 +84,6 @@ Redis should be customized out of its default configuration to secure it
against unauthorized and unauthenticated users. These resources provide
some advice on Reids security and guarding against data breaches.
-* [Pentesting Redis servers](http://averagesecurityguy.info/2015/09/17/pentesting-redis-servers/)
- shows that security is important not only on your application but also
- the databases you're using as well.
-
* Redis, just as with any relational or NoSQL database, needs to be secured
based on [security guidelines](http://www.antirez.com/news/96). There is
also a post where the main author of Redis
diff --git a/content/pages/03-data/12-mongodb.markdown b/content/pages/03-data/12-mongodb.markdown
index 1e9f15161..665144fec 100644
--- a/content/pages/03-data/12-mongodb.markdown
+++ b/content/pages/03-data/12-mongodb.markdown
@@ -99,11 +99,6 @@ security controls so make sure to lock down your instances.
installing and using MongoDB on your own instance. The post covers
authentication, SSL and firewalls.
-* [Securing MongoDB using Let's Encrypt certificate](https://zohaib.me/securing-mongodb-using-lets-encrypt/)
- gives a configuration that encrypts that traffic coming from and
- going to your MongoDB instances using free
- [Let's Encrypt certificates](https://letsencrypt.org/).
-
* This 4 post securing MongoDB series covers
[Data Security Requirements for Regulatory Compliance](https://www.mongodb.com/blog/post/securing-mongodb-part-1-data-security-requirements-for-regulatory-compliance),
[Database Access Control](https://www.mongodb.com/blog/post/securing-mongodb-part-2-database-access-control),
@@ -111,11 +106,6 @@ security controls so make sure to lock down your instances.
and
[Environmental Control & Database Management](https://www.mongodb.com/blog/post/securing-mongodb-part-4-environmental-control-and-database-management).
-* Lightweight Directory Access Protocol (LDAP) is common in many
- established company environments for security. This post on
- [How to Configure LDAP Authentication for MongoDB](https://www.mongodb.com/blog/post/how-to-configure-LDAP-authentication-for-mongodb)
- goes over how to authenticate users via LDAP who are using MongoDB.
-
### Python with MongoDB resources
MongoDB is straightforward to use in a Python application when a driver
diff --git a/content/pages/03-data/16-pandas.markdown b/content/pages/03-data/16-pandas.markdown
index 54cbf13a6..a0750eaae 100644
--- a/content/pages/03-data/16-pandas.markdown
+++ b/content/pages/03-data/16-pandas.markdown
@@ -70,15 +70,18 @@ is a data structures and analysis library.
and charts that show the pay off period broken down by interest and
principal.
+* [Efficiently cleaning text with pandas](https://pbpython.com/text-cleaning.html)
+ provides a really great practical tutorial on different approaches
+ for cleaning a large data set so that you can begin to do your analysis.
+ The tutorial also shows how to use the
+ [sidetable](https://github.com/chris1610/sidetable) library, which
+ creates summary tables of a DataFrame.
+
* [tabula-py: Extract table from PDF into Python DataFrame](https://blog.chezo.uno/tabula-py-extract-table-from-pdf-into-python-dataframe-6c7acfa5f302)
presents how to use the Python wrapper for the
[Tabula](https://tabula.technology/) library that makes it easier to
extract table data from PDF files.
-* [Analyzing Browser History Using Python and Pandas](https://applecrazy.github.io/blog/2017-11-12/analyzing-browser-hist-using-python)
- shows how to take data from Google Chrome and start to visualize it
- with pandas and [matplotlib](/matplotlib.html).
-
* [Time Series Forecast Case Study with Python: Monthly Armed Robberies in Boston](https://machinelearningmastery.com/time-series-forecast-case-study-python-monthly-armed-robberies-boston/)
walks through the data wrangling, analysis and visualization steps
with a public data set of murders in Boston from 1966 to 1975. This
diff --git a/content/pages/03-data/17-scipy-numpy.markdown b/content/pages/03-data/17-scipy-numpy.markdown
index 62ec16355..00763c2d0 100644
--- a/content/pages/03-data/17-scipy-numpy.markdown
+++ b/content/pages/03-data/17-scipy-numpy.markdown
@@ -44,11 +44,6 @@ following resources are broader walkthroughs for the SciPy ecosystem:
plotter. This is a very cool example project that ties together
the scientific world and the art world.
-* [Lectures in Quantitative Economics: SciPy](https://lectures.quantecon.org/py/scipy.html)
- provides a good overview of SciPy compared to the specific NumPy
- project, as well as explanations for the wrappers SciPy provides
- over lower-level FORTRAN libraries.
-
* [A plea for stability in the SciPy ecosystem](http://blog.khinsen.net/posts/2017/11/16/a-plea-for-stability-in-the-scipy-ecosystem/)
presents concerns from one scientist's perspective about how fast the
Python programming ecosystem changes and that code can become backwards
diff --git a/content/pages/03-data/18-data-visualization.markdown b/content/pages/03-data/18-data-visualization.markdown
index af18aada8..e56c2b2dc 100644
--- a/content/pages/03-data/18-data-visualization.markdown
+++ b/content/pages/03-data/18-data-visualization.markdown
@@ -91,11 +91,6 @@ Sometimes you need inspiration from other sources to figure out what
you want to build. The following links have made me excited about data
visualization and gave me ideas for what to build.
-* [Roads to Rome](http://roadstorome.moovellab.com/) is a beautiful
- visualization showing the data behind the expression "all roads lead
- to Rome" and whether or not there is a "Rome" central city in every
- country.
-
* [Monarchs](https://thebackend.dev/monarchs/) is a wonderful 1,000 year
history visual of European rulers. The developer also wrote an in-depth
article on
@@ -107,10 +102,6 @@ visualization and gave me ideas for what to build.
and dark sides, the main characters, various bits about the Force
and other data extracted from the movies.
-* [Big League Graphs](https://bigleaguegraphs.com/) presents a bunch of
- creative ways to view data for sports such as basketball, baseball and
- hockey.
-
* [What do numbers look like?](https://johnhw.github.io/umap_primes/index.md.html)
is a Python 3 dimensional visualization of millions of integers, colored
by special factors such as prime and Fibonacci numbers.
diff --git a/content/pages/03-data/19-bokeh.markdown b/content/pages/03-data/19-bokeh.markdown
index 668538d90..22f8d55cc 100644
--- a/content/pages/03-data/19-bokeh.markdown
+++ b/content/pages/03-data/19-bokeh.markdown
@@ -61,12 +61,6 @@ basic syntax will change as the library's API is not yet stable.
an appropriate format then explains the code that uses Bokeh to visualize
it.
-* [Data is beautiful: Visualizing Roman imperial dynasties](http://machineloveus.com/data-is-beautiful-visualizing-roman-imperial-dynasties/)
- provides a walkthrough for creating a gorgeous visualization based on
- historical Roman data. The post is about more than just the visual, it also
- goes into the ideation, data wrangling and analysis phases that came
- before using Bokeh to show the results.
-
* [Visualizing with Bokeh](https://programminghistorian.org/en/lessons/visualizing-with-bokeh)
gives a detailed explanation with the code for number Bokeh visuals
you can output while working with a [pandas](/pandas.html) data set.
@@ -105,13 +99,6 @@ basic syntax will change as the library's API is not yet stable.
project has the code to create a simple chart with Bokeh and
[Flask](/flask.html).
-* [Bokeh vs Dash — Which is the Best Dashboard Framework for Python?](https://blog.sicara.com/bokeh-dash-best-dashboard-framework-python-shiny-alternative-c5b576375f7f)
- contains a single project that was written in both Dash and Bokeh. The
- author gives his subjective view on the implementation difficulty
- although the web application only contained a single type of data
- visualization so it is hard to drawn any real conclusions from his
- opinion.
-
* [Realtime Flight Tracking with Pandas and Bokeh](https://www.geodose.com/2019/01/realtime-flight-tracking-pandas-bokeh-python.html)
provides a great example of combining [pandas](/pandas.html) for structuring
data with Bokeh for visualization.
diff --git a/content/pages/03-data/21-matplotlib.markdown b/content/pages/03-data/21-matplotlib.markdown
index 14f4d2f64..9c60d3740 100644
--- a/content/pages/03-data/21-matplotlib.markdown
+++ b/content/pages/03-data/21-matplotlib.markdown
@@ -21,10 +21,6 @@ toolkits.
is an awesome getting started tutorial that breaks through the confusing
beginner steps so you can quick start using the plotting library.
-* [Web Scraping With Python: Scrapy, SQL, Matplotlib To Gain Web Data Insights](http://www.scrapingauthority.com/python-scrapy-mysql-and-matplotlib-to-gain-web-data-insights/)
- is a long and comprehensive tutorial that walks through obtaining,
- cleaning and visualizing data.
-
* [Matplotlib Cheat Sheet: Plotting in Python](https://www.datacamp.com/community/blog/python-matplotlib-cheat-sheet)
contains some handy snippets of code to perform common plotting operations
in Matplotlib.
diff --git a/content/pages/03-data/25-oracle.markdown b/content/pages/03-data/25-oracle.markdown
new file mode 100644
index 000000000..bf6b3cdc0
--- /dev/null
+++ b/content/pages/03-data/25-oracle.markdown
@@ -0,0 +1,165 @@
+title: Oracle
+category: page
+slug: Oracle
+sortorder: 0325
+toc: False
+sidebartitle: Oracle
+meta: Oracle Database is an enterprise relational database management system.
+
+
+[Oracle Database](http://www.oracle.com/) is an enterprise
+[relational database](/databases.html). It can run transaction processing,
+data warehousing, and multi-model database workloads such as machine
+learning, spatial, and graph analysis. Recent versions of Oracle Database
+also added support for JSON and blockchain use cases, and the software
+can be run in on-premise, cloud or hybrid environments.
+
+
+
+
+## How does Oracle fit with Python?
+The Python community and Oracle have a long history. The excellent Python Database API-compliant "cx_Oracle" interface for Oracle Database was first created by the user community in 1998 and is now being enhanced and maintained by Oracle. The [cx_Oracle](https://oracle.github.io/python-cx_Oracle/) module also underpins the [Oracle Machine Learning for Python](https://www.youtube.com/watch?v=P861m__PEMQ) engine. Oracle's high-performance GraalVM framework supports an implementation of Python called [GraalPython](https://github.com/oracle/graalpython).
+
+
+## Why is Oracle Database a great choice?
+Oracle Database is cross-platform, supporting multiple hardware platforms and various operating systems. Developers and companies of all sizes rely on its proven industry-leading performance, scalability, reliability, and security.
+As data volumes rise exponentially, new data types and data models are required to support modern applications. Oracle Database supports the following data types at no extra cost:
+
+* [JSON](https://docs.oracle.com/en/database/oracle/oracle-database/19/adjsn/index.html)
+* [Blockchain](https://docs.oracle.com/en/database/oracle/oracle-database/21/nfcon/details-oracle-blockchain-table-282449857.html)
+* [XML](https://www.oracle.com/database/technologies/appdev/xmldb.html)
+* [Object](https://docs.oracle.com/database/121/ADOBJ/adobjint.htm#ADOBJ00101)
+* [Graph](https://www.oracle.com/database/graph/)
+* [Spatial](https://www.oracle.com/database/spatial/)
+* [Time Series](https://docs.oracle.com/en/database/oracle/oracle-database/19/dmcon/time-series.html)
+* Relational
+
+With support for scale-out database clusters, sharded distributed systems, and disaster recovery with continuous application availability, there is no shortage of features to guarantee the Database continues to run uninterrupted 24/7.
+
+Oracle makes its enterprise-class database readily available to developers with its free on-premises edition Oracle Database XE or on the Oracle public cloud with an Always Free Cloud account. In addition, Oracle Autonomous Database is a popular choice for developers as no database management or tuning is required, leaving developers to do what they do best – writing code for their applications.
+
+
+## Connecting to Oracle Database with Python
+As with any database, applications require a connector or driver to connect to the Oracle Database. The Python DB API-compliant [cx_Oracle](https://github.com/oracle/python-cx_Oracle) interface provides developers access to standard and advanced Oracle Database features, such as SQL execution and document storage APIs. It also gives users access to network traffic encryption capabilities and Oracle's leading high availability features.
+
+[Code examples](https://oracle.github.io/python-cx_Oracle/samples/tutorial/Python-and-Oracle-Database-Scripting-for-the-Future.html) and free workshops such as the introductory [Python and Oracle for Developers Workshop](https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=766) and a full-stack development workshop using [Python with SQLAlchemy to Oracle Database](https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=911&clear=180&session=16650643444916) are available.
+
+
+
+You can use many Python frameworks and [object-relational mappers (ORMs)](/object-relational-mappers-orms.html) with Oracle Database. ORMs abstract the tables and objects in a relational database to objects that Python developers can manipulate and operate on. [SQLAlchemy](/sqlalchemy.html) and Django are popular ORMs. SQLAlchemy is used by Pandas, which is very popular with Oracle users.
+The table below shows the relationship between web framework, ORM, driver, and the Oracle Database.
+
+
+
+Learn more about
+[Python ORMs on that dedicated topic page](/object-relational-mappers-orms.html).
+
+ORMs provide a familiar programming model for Python developers, but sometimes you want that extra performance and operate closer to SQL objects. Oracle cx_Oracle offers several [functions](https://oracle.github.io/python-cx_Oracle/samples/tutorial/Python-and-Oracle-Database-Scripting-for-the-Future.html#binding) to deliver that performance. These functions include fetching data, binding data, executing PL/SQL, operating on LOBs, JSON documents, message passing with Oracle Advanced Queuing, and more.
+
+
+## Oracle and Data Safety
+According to Gartner, Oracle has one of the [highest data safety ratings](https://www.gartner.com/reviews/market/cloud-database-management-systems/vendor/oracle/product/oracle-database) in the industry, with a wide range of features for data protection and high availability. These features include:
+
+* [Database encryption](https://www.oracle.com/database/technologies/security/advanced-security.html)
+
+* [Access control to rows](https://www.oracle.com/database/technologies/security/label-security.html) in a table
+
+* [Database vault](https://www.oracle.com/database/technologies/security/db-vault.html) to restrict privileges and access
+
+* [Data redaction, subsetting, and masking](https://www.oracle.com/database/technologies/security/data-masking-subsetting.html)
+
+* All in one data security service in the Oracle Cloud with [Data Safe](https://www.oracle.com/database/technologies/security/data-safe.html)
+
+* Oracle also provides free tools such as the [Database Assessment Tool (DBSAT)](https://www.oracle.com/database/technologies/security/dbsat.html) to help you identify and remedy potential vulnerabilities.
+
+Oracle also provides numerous data recovery features, including:
+
+* Backup capabilities with [RMAN](https://www.oracle.com/database/technologies/high-availability/rman.html)
+
+* Restore point features with [Database Flashback](https://www.oracle.com/database/technologies/high-availability/flashback.html)
+
+* [Application continuity](https://www.oracle.com/database/technologies/high-availability/app-continuity.html) in the event of database failover to a standby
+
+For an overview of Oracle’s security and high availability architecture, see the following white papers:
+
+* [Maximum Availability Architecture](https://www.oracle.com/a/tech/docs/maa-onpremises-overview.pdf) (MAA)
+
+* [Maximum Security Architecture](https://blogs.oracle.com/cloudsecurity/post/oracles-maximum-security-architecture-for-database-security) (MSA)
+
+
+## Python Specific Oracle Database resources
+Many quick starts, tutorials, and workshops exist specifically for Python developers using Oracle Database. Below are some of the best ones to start with.
+
+
+###Getting Started
+If you are looking for a fast way to get started with Python and Oracle Database, check out these two quick start tutorials. These tutorials walk you through installing and setting up the environment you need to connect Python to Oracle Database.
+
+* [Quick Start: Developing Python Applications for Oracle Database](https://www.oracle.com/database/technologies/appdev/python/quickstartpythononprem.html)
+
+* [Quick Start: Developing Python Applications for Oracle Autonomous Database](https://www.oracle.com/database/technologies/appdev/python/quickstartpythononprem.html)
+
+Once you have done one of these, then continue with the popular [Python and Oracle Database Tutorial: Scripting for the Future](https://oracle.github.io/python-cx_Oracle/samples/tutorial/Python-and-Oracle-Database-Scripting-for-the-Future.html) to dive deeper to master the Python cx_Oracle interface and see how to build great Oracle Database applications.
+
+
+###Using Different Frameworks with Oracle
+* [How to Run SQL Queries with Pandas](https://www.oracle.com/news/connect/run-sql-data-queries-with-pandas.html) is a good blog using Pandas for quick and easy data manipulation in Python.
+
+* [Using Oracle with Pandas in OCI Data Science Notebooks](https://docs.oracle.com/en-us/iaas/tools/ads-sdk/latest/user_guide/loading_data/efficient_use_of_oracle_rdbms_with_ads.html) dives deeper into using Pandas with large datasets in data science applications.
+
+* [Using SQLAlchemy with Oracle Database](https://docs.sqlalchemy.org/en/14/dialects/oracle.html) provides an excellent toolkit for Python developers using SQLAlchemy as their ORM.
+
+* [Using Django with Python and Oracle Database](https://www.oracle.com/webfolder/technetwork/tutorials/obe/db/oow10/python_django/python_django.htm) is a tutorial from Oracle and shows the Django Framework with Python to an Oracle Database.
+
+* [Connecting Pony ORM to the Database](https://docs.ponyorm.org/database.html) is a friendly guide on using Pony with databases.
+
+* [How to use Python Flask with Oracle Database](https://blogs.oracle.com/opal/post/how-to-use-python-flask-with-oracle-database).
+
+* [Part 1: Docker for Oracle Database Applications in Node.js and Python](https://blogs.oracle.com/opal/post/part-1-docker-for-oracle-database-applications-in-nodejs-and-python).
+
+* [Part 2: Docker for Oracle Database Applications in Node.js and Python](https://blogs.oracle.com/opal/post/part-2-docker-for-oracle-database-applications-in-nodejs-and-python).
+
+* [Faster JSON with Python cx_Oracle and Oracle Database 21’s new OSON storage format](https://blogs.oracle.com/opal/post/faster-json-with-python-cx_oracle-81-and-oracle-database-21s-new-oson-storage-format).
+
+
+###Workshops
+The following hands-on, free workshops provide step-by-step instructions and walkthroughs in a live environment.
+
+* [Use Python with Oracle Database 19c](https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=635&clear=180&session=3484600041895) is an Oracle LiveLabs workshop that shows how to write Python code to connect to and read data from an Oracle Database, including JSON data.
+
+* [Python and Oracle for Developers](https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/workshop-attendee-2?p210_workshop_id=766&p210_type=2&session=3484600041895) is an Oracle LiveLabs workshop that explores the features of the Python cx_Oracle interface for Oracle Database, including efficient techniques for connection management and statement handling.
+
+* [Full Stack Development using Python and deployment via OKE](https://apexapps.oracle.com/pls/apex/dbpm/r/livelabs/view-workshop?wid=911&clear=180&session=3484600041895) is an Oracle LiveLabs workshop that explores how to build and deploy a simple cloud-native application using the most common frameworks and the Oracle Cloud Infrastructure services.
+
+
+## Cloud Development with Oracle Database
+The following resources are good starting points for those looking to build applications in the Oracle Cloud and deploy applications in Docker containers and Kubernetes.
+
+* [The Complete Guide To Getting Up And Running With Docker And Kubernetes On The Oracle Cloud](https://blogs.oracle.com/developers/post/the-complete-guide-to-getting-up-and-running-with-docker-and-kubernetes-on-the-oracle-cloud).
+
+* [Oracle Cloud Blog](https://www.oc-blog.com/) has lots of interesting information on different aspects of Oracle Cloud.
+
+For developers looking to focus on application development in the Oracle Cloud and not have to worry about managing the Oracle Database, the Autonomous Database is a good choice. All management, including patching and upgrades, scalability, and security, are entirely autonomous. The following resources offer you a glimpse of its capabilities.
+
+* [Julien Dontcheff’s Database Blog](https://juliandontcheff.wordpress.com/category/autonomous/) is a good collection of technical posts with the Autonomous Database.
+
+* [SQL Maria](https://sqlmaria.com/category/autonomous-database/) also has some excellent posts on all things Oracle Database including Autonomous.
+
+* [An Introduction to Autonomous Database](https://questoraclecommunity.org/learn/blogs/oracles-autonomous-database-an-introduction/) gives you a good overview.
+
+* [Autonomous Database for researchers](https://blogs.oracle.com/research/post/a-roadmap-of-oracle-autonomous-database-benefits-for-research) is a good blog with details on some autonomous features.
+
+
+##General Oracle Database Resources
+Here are some Oracle tutorials and resources not specific to Python that can help you take advantage of the Oracle Database features.
+
+* [Oracle Technical Architecture](https://www.oracle.com/webfolder/technetwork/tutorials/architecture-diagrams/18/technical-architecture/database-technical-architecture.html) is from Oracle and has nice visuals and short paragraphs on the architecture of the Oracle Database.
+
+* [Oracle Database Internals](https://databaseinternalmechanism.com/oracle-database-internals/) is an excellent post explaining the architecture of the Oracle Database.
+
+* This [Oracle Performance Tuning](https://blog.quest.com/oracle-performance-tuning-a-5-step-approach-to-optimized-performance/) blog has a 5-step approach to tuning Oracle.
+
+* [Oracle RAC](https://databaseinternalmechanism.com/oracle-rac/) is another good post on the concepts of RAC, Oracle’s Real Application Cluster software for database high availability.
+
+* The [Oracle Database Security](https://www.oracle.com/database/technologies/security.html) web page has lots of information on Oracle’s solutions for security called “defense in depth.”
+
+* This is a good post on the [Top 5 Reasons to choose Oracle](https://www.dbta.com/Editorial/News-Flashes/Top-5-Reasons-to-Use-an-Oracle-Database-144191.aspx) for a production database.
diff --git a/content/pages/04-web-development/00-web-development.markdown b/content/pages/04-web-development/00-web-development.markdown
index 29b3fb98b..919cc6c94 100644
--- a/content/pages/04-web-development/00-web-development.markdown
+++ b/content/pages/04-web-development/00-web-development.markdown
@@ -71,12 +71,6 @@ world.
[data bases](/databases.html), [task queues](/task-queues.html),
[caching](/caching.html) and several other critical concepts.
-* The [Evolution of the Web](http://www.evolutionoftheweb.com/) visualizes
- how web browsers and related technologies have changed over time as well as
- the overall growth of the Internet in the amount of data transferred. Note
- that the visualization unfortunately stops around the beginning of 2013 but
- it's a good way to explore what happened in the first 24 years.
-
* [What happens when?](https://github.com/alex/what-happens-when) is an
incredibly detailed answer to the questions "What happens when you
type google.com into your browser's address box and press enter?" that
@@ -92,19 +86,6 @@ world.
the creation of the URL. This is a great read that provides historical
context for why things are the way they are with the web.
-* [Web app checklist](http://dhilipsiva.com/webapp-checklist/) presents
- good practices that developers building and [deploying](/deployment.html)
- web applications should follow. Don't worry about having every single
- one of these recommendations implemented before getting your site
- live, but it is worthwhile to review the list to make sure there is not
- something obvious you can handle in a few minutes that will improve
- your site's security, performance or usability.
-
-* [Web application development is different and better](http://radar.oreilly.com/2014/01/web-application-development-is-different-and-better.html)
- provides some context for how web development has evolved from writing
- static HTML files into the complex JavaScript client-side applications
- produced today.
-
* [The Browser Hacker's Guide to Instantly Loading Everything](https://www.youtube.com/watch?v=7vUs5yOuv-o)
is a spectacular technical talk given by Addy Osmani at JSConf EU 2017
that has great bits of developer knowledge for both beginner and
diff --git a/content/pages/04-web-development/01-web-frameworks.markdown b/content/pages/04-web-development/01-web-frameworks.markdown
index 6b52e3680..2adde9c83 100644
--- a/content/pages/04-web-development/01-web-frameworks.markdown
+++ b/content/pages/04-web-development/01-web-frameworks.markdown
@@ -90,15 +90,15 @@ but it'll make most developers' lives easier in many cases.
### Comparing web frameworks
+[Talk Python to Me had a podcast episode](https://talkpython.fm/episodes/show/149/4-python-web-frameworks-compared)
+with a detailed comparison of the [Django](/django.html),
+[Flask](/flask.html), Tornado and [Pyramid](/pyramid.html) frameworks.
+
Are you curious about how the code in a Django project is structured compared
with Flask? Check out
[this Django web application tutorial](https://www.twilio.com/docs/sms/tutorials/appointment-reminders-python-django)
and then view [the same application built with Flask](https://www.twilio.com/docs/sms/tutorials/appointment-reminders-python-flask).
-[Talk Python to Me had a podcast episode](https://talkpython.fm/episodes/show/149/4-python-web-frameworks-compared)
-with a detailed comparison of the [Django](/django.html),
-[Flask](/flask.html), Tornado and [Pyramid](/pyramid.html) frameworks.
-
There is also a repository called
[compare-python-web-frameworks](https://github.com/mattmakai/compare-python-web-frameworks)
where the same web application is being coded with varying Python web
@@ -112,6 +112,18 @@ frameworks, templating engines and
and the many other pieces that combine to make web frameworks useful
to web developers.
+* [12 requests per second](https://suade.org/dev/12-requests-per-second-with-python/)
+ examines how the traditionally synchronous web framework
+ [Flask](/flask.html) compares to an async framework like
+ [Sanic](/sanic.html) in an artificial, simple benchmark. The
+ results make it look like Sanic is far faster than Flask, but
+ once you add even a basic amount of functionality to a
+ project, including [databasel](/databases.html) queries
+ and templating, the results even out. Miguel Grinberg
+ also has a great read with broader results in this
+ article asking readers to
+ [Ignore All Web Performance Benchmarks, Including This One](https://blog.miguelgrinberg.com/post/ignore-all-web-performance-benchmarks-including-this-one).
+
* When you are learning how to use one or more web frameworks it's helpful
to have an idea of what the code under the covers is doing. This post on
building a
@@ -150,10 +162,6 @@ frameworks, templating engines and
I agree although I've found sessions and database ORMs to be a helpful
part of a framework when done well.
-* "[What is a web framework?](http://www.jeffknupp.com/blog/2014/03/03/what-is-a-web-framework/)"
- is an in-depth explanation of what web frameworks are and their relation
- to web servers.
-
* [Django vs Flask vs Pyramid: Choosing a Python Web Framework](https://www.airpair.com/python/posts/django-flask-pyramid)
contains background information and code comparisons for similar
web applications built in these three big Python frameworks.
diff --git a/content/pages/04-web-development/02-django.markdown b/content/pages/04-web-development/02-django.markdown
index 79add9f0e..3061d2eee 100644
--- a/content/pages/04-web-development/02-django.markdown
+++ b/content/pages/04-web-development/02-django.markdown
@@ -50,11 +50,6 @@ groups such as [Django District](http://www.meetup.com/django-district/),
[San Francisco Django](http://www.meetup.com/The-San-Francisco-Django-Meetup-Group/)
so new developers can get help when they are stuck.
-There's some debate on whether
-[learning Python by using Django is a bad idea](http://www.jeffknupp.com/blog/2012/12/11/learning-python-via-django-considered-harmful/).
-However, that criticism is invalid if you take the time to learn the Python
-syntax and language semantics first before diving into web development.
-
## Django books and tutorials
There are a slew of free or low cost resources out there for Django. Make
@@ -126,7 +121,7 @@ These books and tutorials assume that you know the basics of building
Django and want to go further to become much more knowledgeable about
the framework.
-* [2 Scoops of Django](https://www.twoscoopspress.com/collections/django/products/two-scoops-of-django-1-11)
+* [2 Scoops of Django](https://www.feldroy.com/books/two-scoops-of-django-3-x)
by Daniel Greenfeld and Audrey Roy is well worth the price of admission if
you're serious about learning how to correctly develop Django websites.
@@ -343,11 +338,6 @@ out how to build your own projects. This is a short list of some
real-world example applications, and many more can be found on the
[Django example projects and code](/django-code-examples.html) page.
-* [Browser calls with Django and Twilio](https://www.twilio.com/docs/howto/walkthrough/browser-calls/python/django)
- shows how to build a web app with Django and
- [Twilio Client](https://www.twilio.com/client) to turn a user's web
- browser into a full-fledged phone. Pretty awesome!
-
* [Openduty](https://github.com/ustream/openduty) is a website status checking
and alert system similar to PagerDuty.
diff --git a/content/pages/04-web-development/03-flask.markdown b/content/pages/04-web-development/03-flask.markdown
index 18c708e54..fd7625f0e 100644
--- a/content/pages/04-web-development/03-flask.markdown
+++ b/content/pages/04-web-development/03-flask.markdown
@@ -205,9 +205,6 @@ combined with the example real-world projects listed in the next section.
is a killer Flask tutorial with all the code needed to create a
web app that can dial phones and receive inbound calls.
-* Jeff Knupp provides some solid advice on how to
- [productionize a Flask app](http://www.jeffknupp.com/blog/2014/01/29/productionizing-a-flask-application/).
-
* If you're looking for a fun tutorial with Flask and WebSockets, check out
my blog post on creating
[Choose Your Own Adventure Presentations with Reveal.js, Python and WebSockets](https://www.twilio.com/blog/2014/11/choose-your-own-adventure-presentations-with-reveal-js-python-and-websockets.html).
@@ -298,7 +295,7 @@ about how to working on your codebase.
* [Bean Counter](https://github.com/BouncyNudibranch/bean-counter) is an
open source Flask app for tracking coffee.
-* [FlaskBB](http://flaskbb.org/) is a Flask app for a discussion forum.
+* [FlaskBB](https://flaskbb.org/) is a Flask app for a discussion forum.
* [psdash](https://github.com/Jahaja/psdash) is an app built with Flask and
psutils to display information about the computer it is running on.
diff --git a/content/pages/04-web-development/04-bottle.markdown b/content/pages/04-web-development/04-bottle.markdown
index 1f40db51c..01e543da4 100644
--- a/content/pages/04-web-development/04-bottle.markdown
+++ b/content/pages/04-web-development/04-bottle.markdown
@@ -29,7 +29,7 @@ Bottle is awesome for a few web development situations:
Prototyping simple ideas is often easier with Bottle than a more
opinionated web framework like [Django](/django.html) because Django
projects start with a significant amount of boilerplate code. The
-[Model-View-Template](https://docs.djangoproject.com/en/1.9/faq/general/#django-appears-to-be-a-mvc-framework-but-you-call-the-controller-the-view-and-the-view-the-template-how-come-you-don-t-use-the-standard-names)
+[Model-View-Template](https://docs.djangoproject.com/en/stable/intro/tutorial03/)
structure for Django apps within projects makes maintaining projects
easier, but it can be cumbersome on starter projects where you're
just playing with random ideas so you aren't worried about your
diff --git a/content/pages/04-web-development/11-template-engines.markdown b/content/pages/04-web-development/11-template-engines.markdown
index f8b7bac06..551a70bfb 100644
--- a/content/pages/04-web-development/11-template-engines.markdown
+++ b/content/pages/04-web-development/11-template-engines.markdown
@@ -161,6 +161,10 @@ know how they work to aid your debugging. The following resources examine
existing template engine design as well as how to build your own engine
when that's necessary for your projects.
+* [Writing a Jinja-inspired template library in Python](https://notes.eatonphil.com/writing-a-template-library-in-python.html)
+ walks through how to create your own a simplified version of the
+ [Jinja](/jinja2.html) template engine as a learning exercise.
+
* [How a template engine works](https://fengsp.github.io/blog/2016/8/how-a-template-engine-works/)
uses the template module in Tornado as an example to step through how
a template engine produces output, from parsing the incoming string to
diff --git a/content/pages/04-web-development/12-jinja2.markdown b/content/pages/04-web-development/12-jinja2.markdown
index 23137b27d..75feb4fdc 100644
--- a/content/pages/04-web-development/12-jinja2.markdown
+++ b/content/pages/04-web-development/12-jinja2.markdown
@@ -8,17 +8,17 @@ meta: Jinja2 is a template engine written in Python for outputting formats such
Jinja, also commonly referred to as
-"[Jinja2](http://jinja.pocoo.org/docs/dev/)" to specify the newest
+"[Jinja2](https://jinja.palletsprojects.com/en/3.0.x/templates/)" to specify the newest
release version, is a Python [template engine](/template-engines.html)
used to create HTML, XML or other markup formats that are returned to the
user via an HTTP response.
-
+
## Why is Jinja2 useful?
Jinja2 is useful because it has consistent template tag syntax and the
project is cleanly extracted as
-[an independent open source project](https://github.com/mitsuhiko/jinja2) so
+[an independent open source project](https://github.com/pallets/jinja) so
it can be used as a dependency by other code libraries.
+
%s" % html)
+
+
+class SearchQueryAdmin(admin.ModelAdmin):
+
+ list_display = (
+ "id",
+ "user",
+ "search_terms_display",
+ "total_hits_display",
+ "returned_",
+ "min_",
+ "max_",
+ "reference",
+ "executed_at",
+ )
+ list_filter = ("index", "query_type")
+ search_fields = ("search_terms", "user__first_name", "user__last_name", "reference")
+ exclude = ("hits", "aggregations", "query", "page", "total_hits_")
+ readonly_fields = (
+ "user",
+ "index",
+ "search_terms",
+ "query_type",
+ "total_hits",
+ "total_hits_relation",
+ "returned_",
+ "min_",
+ "max_",
+ "duration",
+ "query_",
+ "hits_",
+ "aggregations_",
+ "executed_at",
+ )
+
+ def search_terms_display(self, instance: SearchQuery) -> str:
+ raw = instance.search_terms
+~~ return truncatechars(truncatewords(raw, 5), 50)
+
+ def query_(self, instance: SearchQuery) -> str:
+ return pprint(instance.query)
+
+ def max_(self, instance: SearchQuery) -> str:
+ return "-" if instance.page_size == 0 else str(instance.max_score)
+
+ max_.short_description = "Max score" # type: ignore
+
+ def min_(self, instance: SearchQuery) -> str:
+ return "-" if instance.page_size == 0 else str(instance.min_score)
+
+ min_.short_description = "Min score" # type: ignore
+
+ def total_hits_display(self, instance: SearchQuery) -> str:
+ if instance.total_hits_relation == SearchQuery.TotalHitsRelation.ESTIMATE:
+ return f"{instance.total_hits}*"
+ return f"{instance.total_hits}"
+
+ def returned_(self, instance: SearchQuery) -> str:
+ if instance.page_size == 0:
+ return "-"
+ return "%i - %i" % (instance.page_from, instance.page_to)
+
+
+
+## ... source file continues with no further truncatechars examples...
+
+```
+
diff --git a/content/pages/examples/django/django-template-loader-get-template.markdown b/content/pages/examples/django/django-template-loader-get-template.markdown
new file mode 100644
index 000000000..1665b4ca0
--- /dev/null
+++ b/content/pages/examples/django/django-template-loader-get-template.markdown
@@ -0,0 +1,469 @@
+title: django.template.loader get_template Example Code
+category: page
+slug: django-template-loader-get-template-examples
+sortorder: 500011392
+toc: False
+sidebartitle: django.template.loader get_template
+meta: Python example code that shows how to use the get_template callable from the django.template.loader module of the Django project.
+
+
+`get_template` is a callable within the `django.template.loader` module of the Django project.
+
+render_to_string
+and
+select_template
+are a couple of other callables within the `django.template.loader` package that also have code examples.
+
+## Example 1 from dccnsys
+[dccnsys](https://github.com/dccnconf/dccnsys) is a conference registration
+system built with [Django](/django.html). The code is open source under the
+[MIT license](https://github.com/dccnconf/dccnsys/blob/master/LICENSE).
+
+[**dccnsys / wwwdccn / chair_mail / views.py**](https://github.com/dccnconf/dccnsys/blob/master/wwwdccn/chair_mail/views.py)
+
+```python
+# views.py
+from django.contrib import messages
+from django.http import JsonResponse, HttpResponse
+from django.shortcuts import render, get_object_or_404, redirect
+~~from django.template.loader import get_template
+from django.urls import reverse
+from django.utils import timezone
+from django.views.decorators.http import require_GET, require_POST
+
+from conferences.utilities import validate_chair_access
+from chair_mail.context import USER_VARS, CONFERENCE_VARS, SUBMISSION_VARS, \
+ FRAME_VARS
+from chair_mail.forms import EmailFrameUpdateForm, EmailFrameTestForm, \
+ MessageForm, get_preview_form_class, EditNotificationForm, \
+ UpdateNotificationStateForm
+from chair_mail.mailing_lists import ALL_LISTS
+from chair_mail.models import EmailSettings, EmailFrame, EmailMessage, \
+ GroupMessage, MSG_TYPE_USER, MSG_TYPE_SUBMISSION, get_group_message_model, \
+ get_message_leaf_model, SystemNotification, DEFAULT_NOTIFICATIONS_DATA
+from chair_mail.utility import get_email_frame, get_email_frame_or_404, \
+ reverse_preview_url, reverse_list_objects_url, get_object_name, \
+ get_object_url
+from conferences.models import Conference
+
+
+def _get_grouped_vars(msg_type):
+ if msg_type == MSG_TYPE_USER:
+ return (
+ ('Conference variables', CONFERENCE_VARS),
+
+
+## ... source file abbreviated to get to get_template examples ...
+
+
+ ('Submission variables', SUBMISSION_VARS),
+ )
+ raise ValueError(f'unrecognized message type "{msg_type}"')
+
+
+@require_GET
+def overview(request, conf_pk):
+ conference = get_object_or_404(Conference, pk=conf_pk)
+ validate_chair_access(request.user, conference)
+ frame = get_email_frame(conference)
+ return render(request, 'chair_mail/tab_pages/overview.html', context={
+ 'conference': conference,
+ 'frame': frame,
+ 'active_tab': 'overview',
+ })
+
+
+@require_POST
+def create_frame(request, conf_pk):
+ conference = get_object_or_404(Conference, pk=conf_pk)
+ validate_chair_access(request.user, conference)
+ if not hasattr(conference, 'email_settings'):
+ EmailSettings.objects.create(conference=conference)
+ email_settings = conference.email_settings
+ frame = email_settings.frame
+~~ template_html = get_template(
+ 'chair_mail/email/default_frame_html.html').template
+~~ template_plain = get_template(
+ 'chair_mail/email/default_frame_plain.txt').template
+ if frame:
+ frame.text_html = template_html.source
+ frame.text_plain = template_plain.source
+ frame.created_at = timezone.now()
+ frame.updated_at = timezone.now()
+ frame.created_by = request.user
+ frame.save()
+ messages.success(request, 'Reset existing frame')
+ else:
+ frame = EmailFrame.objects.create(
+ conference=conference,
+ created_by=request.user,
+ text_plain=template_plain.source,
+ text_html=template_html.source,
+ )
+ email_settings.frame = frame
+ email_settings.save()
+ messages.success(request, 'Created new template')
+
+ default_next = reverse('chair_mail:overview', kwargs={'conf_pk': conf_pk})
+ next_url = request.GET.get('next', default_next)
+ return redirect(next_url)
+
+
+
+## ... source file continues with no further get_template examples...
+
+```
+
+
+## Example 2 from django-cms
+[django-cms](https://github.com/divio/django-cms)
+([project website](https://www.django-cms.org/en/)) is a Python-based
+content management system (CMS) [library](https://pypi.org/project/django-cms/)
+for use with Django web apps that is open sourced under the
+[BSD 3-Clause "New"](https://github.com/divio/django-cms/blob/develop/LICENSE)
+license.
+
+[**django-cms / cms / templates.py**](https://github.com/divio/django-cms/blob/develop/cms/./templates.py)
+
+```python
+# templates.py
+~~from django.template.loader import get_template
+from django.utils.functional import cached_property
+
+
+class TemplatesCache:
+
+ def __init__(self):
+ self._cached_templates = {}
+
+ def get_cached_template(self, template):
+ if hasattr(template, 'render'):
+ return template
+
+ if not template in self._cached_templates:
+~~ self._cached_templates[template] = get_template(template)
+ return self._cached_templates[template]
+
+ @cached_property
+ def drag_item_template(self):
+~~ return get_template('cms/toolbar/dragitem.html')
+
+ @cached_property
+ def placeholder_plugin_menu_template(self):
+~~ return get_template('cms/toolbar/dragitem_menu.html')
+
+ @cached_property
+ def dragbar_template(self):
+~~ return get_template('cms/toolbar/dragbar.html')
+
+
+
+## ... source file continues with no further get_template examples...
+
+```
+
+
+## Example 3 from django-floppyforms
+[django-floppyforms](https://github.com/jazzband/django-floppyforms)
+([project documentation](https://django-floppyforms.readthedocs.io/en/latest/)
+and
+[PyPI page](https://pypi.org/project/django-floppyforms/))
+is a [Django](/django.html) code library for better control
+over rendering HTML forms in your [templates](/template-engines.html).
+
+The django-floppyforms code is provided as
+[open source](https://github.com/jazzband/django-floppyforms/blob/master/LICENSE)
+and maintained by the collaborative developer community group
+[Jazzband](https://jazzband.co/).
+
+[**django-floppyforms / floppyforms / compat.py**](https://github.com/jazzband/django-floppyforms/blob/master/floppyforms/./compat.py)
+
+```python
+# compat.py
+from contextlib import contextmanager
+
+import django
+from django.template import Context
+from django.utils.datastructures import MultiValueDict
+
+MULTIVALUE_DICT_TYPES = (MultiValueDict,)
+
+
+REQUIRED_CONTEXT_ATTRIBTUES = (
+ '_form_config',
+ '_form_render',
+)
+
+
+class DictContext(dict):
+ pass
+
+
+if django.VERSION < (1, 8):
+~~ def get_template(context, template_name):
+~~ from django.template.loader import get_template
+~~ return get_template(template_name)
+
+ def get_context(context):
+ if not isinstance(context, Context):
+ context = Context(context)
+ return context
+
+else:
+~~ def get_template(context, template_name):
+ return context.template.engine.get_template(template_name)
+
+ def get_context(context):
+ return context
+
+
+def flatten_context(context):
+ if isinstance(context, Context):
+ flat = {}
+ for d in context.dicts:
+ flat.update(d)
+ return flat
+ else:
+ return context
+
+
+def flatten_contexts(*contexts):
+ new_context = DictContext()
+ for context in contexts:
+ if context is not None:
+ new_context.update(flatten_context(context))
+ for attr in REQUIRED_CONTEXT_ATTRIBTUES:
+ if hasattr(context, attr):
+ setattr(new_context, attr, getattr(context, attr))
+
+
+## ... source file continues with no further get_template examples...
+
+```
+
+
+## Example 4 from django-sitetree
+[django-sitetree](https://github.com/idlesign/django-sitetree)
+([project documentation](https://django-sitetree.readthedocs.io/en/latest/)
+and
+[PyPI package information](https://pypi.org/project/django-sitetree/))
+is a [Django](/django.html) extension that makes it easier for
+developers to add site trees, menus and breadcrumb navigation elements
+to their web applications.
+
+The django-sitetree project is provided as open source under the
+[BSD 3-Clause "New" or "Revised" License](https://github.com/idlesign/django-sitetree/blob/master/LICENSE).
+
+[**django-sitetree / sitetree / sitetreeapp.py**](https://github.com/idlesign/django-sitetree/blob/master/sitetree/./sitetreeapp.py)
+
+```python
+# sitetreeapp.py
+import warnings
+from collections import defaultdict
+from copy import deepcopy
+from inspect import getfullargspec
+from sys import exc_info
+from threading import local
+from typing import Callable, List, Optional, Dict, Union, Sequence, Any, Tuple
+
+from django.conf import settings
+from django.core.cache import caches
+from django.db.models import signals, QuerySet
+from django.template.base import (
+ FilterExpression, Lexer, Parser, Variable, VariableDoesNotExist, VARIABLE_TAG_START)
+from django.template.context import Context
+~~from django.template.loader import get_template
+from django.urls import reverse, NoReverseMatch
+from django.utils import module_loading
+from django.utils.encoding import iri_to_uri
+from django.utils.translation import get_language
+
+from .compat import TOKEN_TEXT, TOKEN_VAR
+from .exceptions import SiteTreeError
+from .settings import (
+ ALIAS_TRUNK, ALIAS_THIS_CHILDREN, ALIAS_THIS_SIBLINGS, ALIAS_THIS_PARENT_SIBLINGS, ALIAS_THIS_ANCESTOR_CHILDREN,
+ UNRESOLVED_ITEM_MARKER, RAISE_ITEMS_ERRORS_ON_DEBUG, CACHE_TIMEOUT, CACHE_NAME, DYNAMIC_ONLY, ADMIN_APP_NAME,
+ SITETREE_CLS)
+from .utils import get_tree_model, get_tree_item_model, import_app_sitetree_module, generate_id_for
+
+if False: # pragma: nocover
+ from django.contrib.auth.models import User # noqa
+ from .models import TreeItemBase, TreeBase
+
+TypeDynamicTrees = Dict[str, Union[Dict[str, List['TreeBase']], List['TreeBase']]]
+
+MODEL_TREE_CLASS = get_tree_model()
+MODEL_TREE_ITEM_CLASS = get_tree_item_model()
+
+
+_ITEMS_PROCESSOR: Optional[Callable] = None
+
+
+## ... source file abbreviated to get to get_template examples ...
+
+
+ return []
+
+ tree_items = self.filter_items(self.get_children(tree_alias, None), 'sitetree')
+ tree_items = self.apply_hook(tree_items, 'sitetree')
+ self.update_has_children(tree_alias, tree_items, 'sitetree')
+
+ return tree_items
+
+ def children(
+ self,
+ parent_item: 'TreeItemBase',
+ navigation_type: str,
+ use_template: str,
+ context: Context
+ ) -> str:
+ parent_item = self.resolve_var(parent_item, context)
+ tree_alias, tree_items = self.get_sitetree(parent_item.tree.alias)
+
+ self.tree_climber(tree_alias, self.get_tree_current_item(tree_alias))
+
+ tree_items = self.get_children(tree_alias, parent_item)
+ tree_items = self.filter_items(tree_items, navigation_type)
+ tree_items = self.apply_hook(tree_items, f'{navigation_type}.children')
+ self.update_has_children(tree_alias, tree_items, navigation_type)
+
+~~ my_template = get_template(use_template)
+
+ context.push()
+ context['sitetree_items'] = tree_items
+ rendered = my_template.render(context.flatten())
+ context.pop()
+
+ return rendered
+
+ def get_children(self, tree_alias: str, item: Optional['TreeItemBase']) -> List['TreeItemBase']:
+ if not self._current_app_is_admin:
+ tree_alias = self.resolve_tree_i18n_alias(tree_alias)
+
+ return self.cache.get_entry('parents', tree_alias)[item]
+
+ def update_has_children(self, tree_alias: str, tree_items: List['TreeItemBase'], navigation_type: str):
+ get_children = self.get_children
+ filter_items = self.filter_items
+ apply_hook = self.apply_hook
+
+ for tree_item in tree_items:
+ children = get_children(tree_alias, tree_item)
+ children = filter_items(children, navigation_type)
+ children = apply_hook(children, f'{navigation_type}.has_children')
+ tree_item.has_children = len(children) > 0
+
+
+## ... source file continues with no further get_template examples...
+
+```
+
+
+## Example 5 from django-tables2
+[django-tables2](https://github.com/jieter/django-tables2)
+([projection documentation](https://django-tables2.readthedocs.io/en/latest/)
+and
+[PyPI page](https://pypi.org/project/django-tables2/))
+is a code library for [Django](/django.html) that simplifies creating and
+displaying tables in [Django templates](/django-templates.html),
+especially with more advanced features such as pagination and sorting.
+The project and its code are
+[available as open source](https://github.com/jieter/django-tables2/blob/master/LICENSE).
+
+[**django-tables2 / django_tables2 / tables.py**](https://github.com/jieter/django-tables2/blob/master/django_tables2/./tables.py)
+
+```python
+# tables.py
+import copy
+from collections import OrderedDict
+from itertools import count
+
+from django.conf import settings
+from django.core.paginator import Paginator
+from django.db import models
+~~from django.template.loader import get_template
+from django.utils.encoding import force_str
+
+from . import columns
+from .config import RequestConfig
+from .data import TableData
+from .rows import BoundRows
+from .utils import Accessor, AttributeDict, OrderBy, OrderByTuple, Sequence
+
+
+class DeclarativeColumnsMetaclass(type):
+
+ def __new__(mcs, name, bases, attrs):
+ attrs["_meta"] = opts = TableOptions(attrs.get("Meta", None), name)
+
+ cols, remainder = [], {}
+ for attr_name, attr in attrs.items():
+ if isinstance(attr, columns.Column):
+ attr._explicit = True
+ cols.append((attr_name, attr))
+ else:
+ remainder[attr_name] = attr
+ attrs = remainder
+
+ cols.sort(key=lambda x: x[1].creation_counter)
+
+
+## ... source file abbreviated to get to get_template examples ...
+
+
+ order_by = self._meta.order_by
+ if order_by is None:
+ self._order_by = None
+ order_by = self.data.ordering
+ if order_by is not None:
+ self.order_by = order_by
+ else:
+ self.order_by = order_by
+ self.template_name = template_name
+ if request:
+ RequestConfig(request).configure(self)
+
+ self._counter = count()
+
+ def get_top_pinned_data(self):
+ return None
+
+ def get_bottom_pinned_data(self):
+ return None
+
+ def before_render(self, request):
+ return
+
+ def as_html(self, request):
+ self._counter = count()
+~~ template = get_template(self.template_name)
+
+ context = {"table": self, "request": request}
+
+ self.before_render(request)
+ return template.render(context)
+
+ def as_values(self, exclude_columns=None):
+ if exclude_columns is None:
+ exclude_columns = ()
+
+ columns = [
+ column
+ for column in self.columns.iterall()
+ if not (column.column.exclude_from_export or column.name in exclude_columns)
+ ]
+
+ yield [force_str(column.header, strings_only=True) for column in columns]
+
+ for row in self.rows:
+ yield [
+ force_str(row.get_cell_value(column.name), strings_only=True) for column in columns
+ ]
+
+ def has_footer(self):
+
+
+## ... source file continues with no further get_template examples...
+
+```
+
diff --git a/content/pages/examples/django/django-template-loader-render-to-string.markdown b/content/pages/examples/django/django-template-loader-render-to-string.markdown
new file mode 100644
index 000000000..ac9390c29
--- /dev/null
+++ b/content/pages/examples/django/django-template-loader-render-to-string.markdown
@@ -0,0 +1,1293 @@
+title: django.template.loader render_to_string Example Code
+category: page
+slug: django-template-loader-render-to-string-examples
+sortorder: 500011393
+toc: False
+sidebartitle: django.template.loader render_to_string
+meta: Python example code that shows how to use the render_to_string callable from the django.template.loader module of the Django project.
+
+
+`render_to_string` is a callable within the `django.template.loader` module of the Django project.
+
+get_template
+and
+select_template
+are a couple of other callables within the `django.template.loader` package that also have code examples.
+
+## Example 1 from dccnsys
+[dccnsys](https://github.com/dccnconf/dccnsys) is a conference registration
+system built with [Django](/django.html). The code is open source under the
+[MIT license](https://github.com/dccnconf/dccnsys/blob/master/LICENSE).
+
+[**dccnsys / wwwdccn / auth_app / views.py**](https://github.com/dccnconf/dccnsys/blob/master/wwwdccn/auth_app/views.py)
+
+```python
+# views.py
+import json
+from urllib.request import Request, urlopen
+from urllib.parse import urlencode
+
+from django.conf import settings
+from django.contrib.auth import views as auth_views
+from django.contrib.auth import get_user_model, login
+from django.shortcuts import redirect, render
+~~from django.template.loader import render_to_string
+from django.core.mail import send_mail
+
+from .forms import SignUpForm
+
+User = get_user_model()
+
+
+def signup(request):
+ if request.method == 'POST':
+ form = SignUpForm(request.POST)
+ if form.is_valid():
+ recaptcha_response = request.POST.get('g-recaptcha-response')
+ url = 'https://www.google.com/recaptcha/api/siteverify'
+ values = {
+ 'secret': settings.RECAPTCHA_SECRET_KEY,
+ 'response': recaptcha_response
+ }
+ data = urlencode(values).encode()
+ req = Request(url, data=data)
+
+ response = urlopen(req)
+ result = json.loads(response.read().decode())
+ if result['success']:
+ user = form.save()
+ user.is_active = True
+ user.save()
+ login(request, user)
+
+ context = {
+ 'email': user.email,
+ 'protocol': 'https' if request.is_secure() else "http",
+ 'domain': request.get_host(),
+ }
+~~ html = render_to_string('auth_app/email/welcome.html', context)
+~~ text = render_to_string('auth_app/email/welcome.txt', context)
+ send_mail(
+ 'Welcome to DCCN Conference Registration System!',
+ message=text,
+ html_message=html,
+ recipient_list=[user.email],
+ from_email=settings.DEFAULT_FROM_EMAIL,
+ fail_silently=False,
+ )
+ return redirect('register')
+ else:
+ form = SignUpForm()
+ return render(request, 'auth_app/signup.html', {
+ 'site_key': settings.RECAPTCHA_SITE_KEY,
+ 'form': form,
+ })
+
+
+class PasswordResetDoneView(auth_views.PasswordResetDoneView):
+ template_name = 'auth_app/password_reset_done.html'
+
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 2 from django-allauth
+[django-allauth](https://github.com/pennersr/django-allauth)
+([project website](https://www.intenct.nl/projects/django-allauth/)) is a
+[Django](/django.html) library for easily adding local and social authentication
+flows to Django projects. It is open source under the
+[MIT License](https://github.com/pennersr/django-allauth/blob/master/LICENSE).
+
+
+[**django-allauth / allauth / account / adapter.py**](https://github.com/pennersr/django-allauth/blob/master/allauth/account/adapter.py)
+
+```python
+# adapter.py
+from __future__ import unicode_literals
+
+import hashlib
+import json
+import time
+import warnings
+
+from django import forms
+from django.conf import settings
+from django.contrib import messages
+from django.contrib.auth import (
+ authenticate,
+ get_backends,
+ login as django_login,
+ logout as django_logout,
+)
+from django.contrib.auth.models import AbstractUser
+from django.contrib.auth.password_validation import validate_password
+from django.contrib.sites.shortcuts import get_current_site
+from django.core.cache import cache
+from django.core.mail import EmailMessage, EmailMultiAlternatives
+from django.http import HttpResponse, HttpResponseRedirect
+from django.shortcuts import resolve_url
+from django.template import TemplateDoesNotExist
+~~from django.template.loader import render_to_string
+from django.urls import reverse
+from django.utils import timezone
+from django.utils.encoding import force_str
+from django.utils.translation import gettext_lazy as _
+
+from ..utils import (
+ build_absolute_uri,
+ email_address_exists,
+ generate_unique_username,
+ get_user_model,
+ import_attribute,
+)
+from . import app_settings
+
+
+class DefaultAccountAdapter(object):
+
+ error_messages = {
+ "username_blacklisted": _(
+ "Username can not be used. Please use other username."
+ ),
+ "username_taken": AbstractUser._meta.get_field("username").error_messages[
+ "unique"
+ ],
+
+
+## ... source file abbreviated to get to render_to_string examples ...
+
+
+ def stash_user(self, request, user):
+ request.session["account_user"] = user
+
+ def unstash_user(self, request):
+ return request.session.pop("account_user", None)
+
+ def is_email_verified(self, request, email):
+ ret = False
+ verified_email = request.session.get("account_verified_email")
+ if verified_email:
+ ret = verified_email.lower() == email.lower()
+ return ret
+
+ def format_email_subject(self, subject):
+ prefix = app_settings.EMAIL_SUBJECT_PREFIX
+ if prefix is None:
+ site = get_current_site(self.request)
+ prefix = "[{name}] ".format(name=site.name)
+ return prefix + force_str(subject)
+
+ def get_from_email(self):
+ return settings.DEFAULT_FROM_EMAIL
+
+ def render_mail(self, template_prefix, email, context):
+ to = [email] if isinstance(email, str) else email
+~~ subject = render_to_string("{0}_subject.txt".format(template_prefix), context)
+ subject = " ".join(subject.splitlines()).strip()
+ subject = self.format_email_subject(subject)
+
+ from_email = self.get_from_email()
+
+ bodies = {}
+ for ext in ["html", "txt"]:
+ try:
+ template_name = "{0}_message.{1}".format(template_prefix, ext)
+~~ bodies[ext] = render_to_string(
+ template_name,
+ context,
+ self.request,
+ ).strip()
+ except TemplateDoesNotExist:
+ if ext == "txt" and not bodies:
+ raise
+ if "txt" in bodies:
+ msg = EmailMultiAlternatives(subject, bodies["txt"], from_email, to)
+ if "html" in bodies:
+ msg.attach_alternative(bodies["html"], "text/html")
+ else:
+ msg = EmailMessage(subject, bodies["html"], from_email, to)
+ msg.content_subtype = "html" # Main content is now text/html
+ return msg
+
+ def send_mail(self, template_prefix, email, context):
+ msg = self.render_mail(template_prefix, email, context)
+ msg.send()
+
+ def get_signup_redirect_url(self, request):
+ return resolve_url(app_settings.SIGNUP_REDIRECT_URL)
+
+ def get_login_redirect_url(self, request):
+
+
+## ... source file abbreviated to get to render_to_string examples ...
+
+
+ min_length = app_settings.PASSWORD_MIN_LENGTH
+ if min_length and len(password) < min_length:
+ raise forms.ValidationError(
+ _("Password must be a minimum of {0} " "characters.").format(min_length)
+ )
+ validate_password(password, user)
+ return password
+
+ def validate_unique_email(self, email):
+ if email_address_exists(email):
+ raise forms.ValidationError(self.error_messages["email_taken"])
+ return email
+
+ def add_message(
+ self,
+ request,
+ level,
+ message_template,
+ message_context=None,
+ extra_tags="",
+ ):
+ if "django.contrib.messages" in settings.INSTALLED_APPS:
+ try:
+ if message_context is None:
+ message_context = {}
+~~ message = render_to_string(
+ message_template,
+ message_context,
+ self.request,
+ ).strip()
+ if message:
+ messages.add_message(request, level, message, extra_tags=extra_tags)
+ except TemplateDoesNotExist:
+ pass
+
+ def ajax_response(self, request, response, redirect_to=None, form=None, data=None):
+ resp = {}
+ status = response.status_code
+
+ if redirect_to:
+ status = 200
+ resp["location"] = redirect_to
+ if form:
+ if request.method == "POST":
+ if form.is_valid():
+ status = 200
+ else:
+ status = 400
+ else:
+ status = 200
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 3 from django-cms
+[django-cms](https://github.com/divio/django-cms)
+([project website](https://www.django-cms.org/en/)) is a Python-based
+content management system (CMS) [library](https://pypi.org/project/django-cms/)
+for use with Django web apps that is open sourced under the
+[BSD 3-Clause "New"](https://github.com/divio/django-cms/blob/develop/LICENSE)
+license.
+
+[**django-cms / cms / templatetags / cms_tags.py**](https://github.com/divio/django-cms/blob/develop/cms/templatetags/cms_tags.py)
+
+```python
+# cms_tags.py
+from collections import namedtuple, OrderedDict
+from copy import copy
+from datetime import datetime
+
+from django import template
+from django.conf import settings
+from django.contrib.sites.models import Site
+from django.core.mail import mail_managers
+from django.db.models import Model
+from django.middleware.common import BrokenLinkEmailsMiddleware
+~~from django.template.loader import render_to_string
+from django.urls import reverse
+from django.utils.encoding import force_text, smart_text
+from django.utils.html import escape
+from django.utils.http import urlencode
+from django.utils.translation import (
+ get_language,
+ override as force_language,
+ gettext_lazy as _,
+)
+
+from classytags.arguments import (Argument, MultiValueArgument,
+ MultiKeywordArgument)
+from classytags.core import Options, Tag
+from classytags.helpers import InclusionTag, AsTag
+from classytags.parser import Parser
+from classytags.utils import flatten_context
+from classytags.values import ListValue, StringValue
+
+from cms.cache.page import get_page_url_cache, set_page_url_cache
+from cms.exceptions import PlaceholderNotFound
+from cms.models import Page, Placeholder as PlaceholderModel, CMSPlugin, StaticPlaceholder
+from cms.plugin_pool import plugin_pool
+from cms.toolbar.utils import get_toolbar_from_request
+from cms.utils import get_current_site, get_language_from_request, get_site_id
+
+
+## ... source file abbreviated to get to render_to_string examples ...
+
+
+ Argument('edit_fields', default=None, required=False),
+ Argument('language', default=None, required=False),
+ Argument('filters', default=None, required=False),
+ Argument('view_url', default=None, required=False),
+ Argument('view_method', default=None, required=False),
+ 'as',
+ Argument('varname', required=False, resolve=False),
+ )
+
+ def __init__(self, parser, tokens):
+ self.parser = parser
+ super().__init__(parser, tokens)
+
+ def _is_editable(self, request):
+ return (request and hasattr(request, 'toolbar') and request.toolbar.edit_mode_active)
+
+ def get_template(self, context, **kwargs):
+ if self._is_editable(context.get('request', None)):
+ return self.edit_template
+ return self.template
+
+ def render_tag(self, context, **kwargs):
+ context.push()
+ template = self.get_template(context, **kwargs)
+ data = self.get_context(context, **kwargs)
+~~ output = render_to_string(template, flatten_context(data)).strip()
+ context.pop()
+ if kwargs.get('varname'):
+ context[kwargs['varname']] = output
+ return ''
+ else:
+ return output
+
+ def _get_editable_context(self, context, instance, language, edit_fields,
+ view_method, view_url, querystring, editmode=True):
+ request = context['request']
+ if hasattr(request, 'toolbar'):
+ lang = request.toolbar.toolbar_language
+ else:
+ lang = get_language()
+ opts = instance._meta
+ if getattr(instance, '_deferred', False):
+ opts = opts.proxy_for_model._meta
+ with force_language(lang):
+ extra_context = {}
+ if edit_fields == 'changelist':
+ instance.get_plugin_name = u"%s %s list" % (smart_text(_('Edit')), smart_text(opts.verbose_name))
+ extra_context['attribute_name'] = 'changelist'
+ elif editmode:
+ instance.get_plugin_name = u"%s %s" % (smart_text(_('Edit')), smart_text(opts.verbose_name))
+
+
+## ... source file abbreviated to get to render_to_string examples ...
+
+
+ extra_context = self._get_empty_context(context, instance, None,
+ language, view_url,
+ view_method, editmode=False)
+ extra_context['render_model_add'] = True
+ return extra_context
+
+
+class CMSEditableObjectAddBlock(CMSEditableObject):
+ name = 'render_model_add_block'
+ options = Options(
+ Argument('instance'),
+ Argument('language', default=None, required=False),
+ Argument('view_url', default=None, required=False),
+ Argument('view_method', default=None, required=False),
+ 'as',
+ Argument('varname', required=False, resolve=False),
+ blocks=[('endrender_model_add_block', 'nodelist')],
+ )
+
+ def render_tag(self, context, **kwargs):
+ context.push()
+ template = self.get_template(context, **kwargs)
+ data = self.get_context(context, **kwargs)
+ data['content'] = kwargs['nodelist'].render(data)
+ data['rendered_content'] = data['content']
+~~ output = render_to_string(template, flatten_context(data))
+ context.pop()
+ if kwargs.get('varname'):
+ context[kwargs['varname']] = output
+ return ''
+ else:
+ return output
+
+ def get_context(self, context, **kwargs):
+ instance = kwargs.pop('instance')
+ if isinstance(instance, Model) and not instance.pk:
+ instance.pk = 0
+ kwargs.pop('varname')
+ kwargs.pop('nodelist')
+ extra_context = self._get_empty_context(context, instance, None,
+ editmode=False, **kwargs)
+ extra_context['render_model_add'] = True
+ return extra_context
+
+
+class CMSEditableObjectBlock(CMSEditableObject):
+ name = 'render_model_block'
+ options = Options(
+ Argument('instance'),
+ Argument('edit_fields', default=None, required=False),
+ Argument('language', default=None, required=False),
+ Argument('view_url', default=None, required=False),
+ Argument('view_method', default=None, required=False),
+ 'as',
+ Argument('varname', required=False, resolve=False),
+ blocks=[('endrender_model_block', 'nodelist')],
+ )
+
+ def render_tag(self, context, **kwargs):
+ context.push()
+ template = self.get_template(context, **kwargs)
+ data = self.get_context(context, **kwargs)
+ data['content'] = kwargs['nodelist'].render(data)
+ data['rendered_content'] = data['content']
+~~ output = render_to_string(template, flatten_context(data))
+ context.pop()
+ if kwargs.get('varname'):
+ context[kwargs['varname']] = output
+ return ''
+ else:
+ return output
+
+ def get_context(self, context, **kwargs):
+ kwargs.pop('varname')
+ kwargs.pop('nodelist')
+ extra_context = self._get_empty_context(context, **kwargs)
+ extra_context['instance'] = kwargs.get('instance')
+ extra_context['render_model_block'] = True
+ return extra_context
+
+
+class StaticPlaceholderNode(Tag):
+ name = 'static_placeholder'
+ options = PlaceholderOptions(
+ Argument('code', required=True),
+ MultiValueArgument('extra_bits', required=False, resolve=False),
+ blocks=[
+ ('endstatic_placeholder', 'nodelist'),
+ ]
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 4 from django-debug-toolbar
+[django-debug-toolbar](https://github.com/jazzband/django-debug-toolbar)
+([project documentation](https://github.com/jazzband/django-debug-toolbar)
+and [PyPI page](https://pypi.org/project/django-debug-toolbar/))
+grants a developer detailed request-response cycle information while
+developing a [Django](/django.html) web application.
+The code for django-debug-toolbar is
+[open source](https://github.com/jazzband/django-debug-toolbar/blob/master/LICENSE)
+and maintained by the developer community group known as
+[Jazzband](https://jazzband.co/).
+
+[**django-debug-toolbar / debug_toolbar / toolbar.py**](https://github.com/jazzband/django-debug-toolbar/blob/master/debug_toolbar/./toolbar.py)
+
+```python
+# toolbar.py
+
+import uuid
+from collections import OrderedDict
+
+from django.apps import apps
+from django.core.exceptions import ImproperlyConfigured
+from django.template import TemplateSyntaxError
+~~from django.template.loader import render_to_string
+from django.urls import path
+from django.utils.module_loading import import_string
+
+from debug_toolbar import settings as dt_settings
+
+
+class DebugToolbar:
+ def __init__(self, request, get_response):
+ self.request = request
+ self.config = dt_settings.get_config().copy()
+ panels = []
+ for panel_class in reversed(self.get_panel_classes()):
+ panel = panel_class(self, get_response)
+ panels.append(panel)
+ if panel.enabled:
+ get_response = panel.process_request
+ self.process_request = get_response
+ self._panels = OrderedDict()
+ while panels:
+ panel = panels.pop()
+ self._panels[panel.panel_id] = panel
+ self.stats = {}
+ self.server_timing_stats = {}
+ self.store_id = None
+
+
+ @property
+ def panels(self):
+ return list(self._panels.values())
+
+ @property
+ def enabled_panels(self):
+ return [panel for panel in self._panels.values() if panel.enabled]
+
+ def get_panel_by_id(self, panel_id):
+ return self._panels[panel_id]
+
+
+ def render_toolbar(self):
+ if not self.should_render_panels():
+ self.store()
+ try:
+ context = {"toolbar": self}
+~~ return render_to_string("debug_toolbar/base.html", context)
+ except TemplateSyntaxError:
+ if not apps.is_installed("django.contrib.staticfiles"):
+ raise ImproperlyConfigured(
+ "The debug toolbar requires the staticfiles contrib app. "
+ "Add 'django.contrib.staticfiles' to INSTALLED_APPS and "
+ "define STATIC_URL in your settings."
+ )
+ else:
+ raise
+
+ def should_render_panels(self):
+ render_panels = self.config["RENDER_PANELS"]
+ if render_panels is None:
+ render_panels = self.request.META["wsgi.multiprocess"]
+ return render_panels
+
+
+ _store = OrderedDict()
+
+ def store(self):
+ if self.store_id:
+ return
+ self.store_id = uuid.uuid4().hex
+ self._store[self.store_id] = self
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 5 from django-filer
+[django-filer](https://github.com/divio/django-filer)
+([project documentation](https://django-filer.readthedocs.io/en/latest/))
+is a file management library for uploading and organizing files and images
+in Django's admin interface. The project's code is available under the
+[BSD 3-Clause "New" or "Revised" open source license](https://github.com/divio/django-filer/blob/develop/LICENSE.txt).
+
+[**django-filer / filer / fields / folder.py**](https://github.com/divio/django-filer/blob/develop/filer/fields/folder.py)
+
+```python
+# folder.py
+from __future__ import absolute_import
+
+import warnings
+
+from django import forms
+from django.contrib.admin.sites import site
+from django.contrib.admin.widgets import ForeignKeyRawIdWidget
+from django.core.exceptions import ObjectDoesNotExist
+from django.db import models
+~~from django.template.loader import render_to_string
+from django.urls import reverse
+from django.utils.http import urlencode
+from django.utils.safestring import mark_safe
+
+from ..models import Folder
+from ..utils.compatibility import truncate_words
+from ..utils.model_label import get_model_label
+
+
+class AdminFolderWidget(ForeignKeyRawIdWidget):
+ choices = None
+ input_type = 'hidden'
+ is_hidden = False
+
+ def render(self, name, value, attrs=None, renderer=None):
+ obj = self.obj_for_value(value)
+ css_id = attrs.get('id')
+ css_id_folder = "%s_folder" % css_id
+ css_id_description_txt = "%s_description_txt" % css_id
+ if attrs is None:
+ attrs = {}
+ related_url = None
+
+ if value:
+
+
+## ... source file abbreviated to get to render_to_string examples ...
+
+
+ if not related_url:
+ related_url = reverse('admin:filer-directory_listing-last')
+ params = self.url_parameters()
+ params['_pick'] = 'folder'
+ if params:
+ url = '?' + urlencode(sorted(params.items()))
+ else:
+ url = ''
+ if 'class' not in attrs:
+ attrs['class'] = 'vForeignKeyRawIdAdminField'
+ super_attrs = attrs.copy()
+ hidden_input = super(ForeignKeyRawIdWidget, self).render(name, value, super_attrs)
+
+ context = {
+ 'hidden_input': hidden_input,
+ 'lookup_url': '%s%s' % (related_url, url),
+ 'lookup_name': name,
+ 'span_id': css_id_description_txt,
+ 'object': obj,
+ 'clear_id': '%s_clear' % css_id,
+ 'descid': css_id_description_txt,
+ 'noimg': 'filer/icons/nofile_32x32.png',
+ 'foldid': css_id_folder,
+ 'id': css_id,
+ }
+~~ html = render_to_string('admin/filer/widgets/admin_folder.html', context)
+ return mark_safe(html)
+
+ def label_for_value(self, value):
+ obj = self.obj_for_value(value)
+ return ' %s' % truncate_words(obj, 14)
+
+ def obj_for_value(self, value):
+ if not value:
+ return None
+ try:
+ key = self.rel.get_related_field().name
+ obj = self.rel.model._default_manager.get(**{key: value})
+ except ObjectDoesNotExist:
+ obj = None
+ return obj
+
+ class Media(object):
+ js = (
+ 'filer/js/addons/popup_handling.js',
+ )
+
+
+class AdminFolderFormField(forms.ModelChoiceField):
+ widget = AdminFolderWidget
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 6 from django-haystack
+[django-haystack](https://github.com/django-haystack/django-haystack)
+([project website](http://haystacksearch.org/) and
+[PyPI page](https://pypi.org/project/django-haystack/))
+is a search abstraction layer that separates the Python search code
+in a [Django](/django.html) web application from the search engine
+implementation that it runs on, such as
+[Apache Solr](http://lucene.apache.org/solr/),
+[Elasticsearch](https://www.elastic.co/)
+or [Whoosh](https://whoosh.readthedocs.io/en/latest/intro.html).
+
+The django-haystack project is open source under the
+[BSD license](https://github.com/django-haystack/django-haystack/blob/master/LICENSE).
+
+[**django-haystack / haystack / panels.py**](https://github.com/django-haystack/django-haystack/blob/master/haystack/./panels.py)
+
+```python
+# panels.py
+import datetime
+
+from debug_toolbar.panels import DebugPanel
+~~from django.template.loader import render_to_string
+from django.utils.translation import ugettext_lazy as _
+
+from haystack import connections
+
+
+class HaystackDebugPanel(DebugPanel):
+
+ name = "Haystack"
+ has_content = True
+
+ def __init__(self, *args, **kwargs):
+ super(self.__class__, self).__init__(*args, **kwargs)
+ self._offset = dict(
+ (alias, len(connections[alias].queries))
+ for alias in connections.connections_info.keys()
+ )
+ self._search_time = 0
+ self._queries = []
+ self._backends = {}
+
+ def nav_title(self):
+ return _("Haystack")
+
+ def nav_subtitle(self):
+
+
+## ... source file abbreviated to get to render_to_string examples ...
+
+
+ if query.get("additional_kwargs"):
+ if query["additional_kwargs"].get("result_class"):
+ query["additional_kwargs"]["result_class"] = str(
+ query["additional_kwargs"]["result_class"]
+ )
+
+ try:
+ query["width_ratio"] = (float(query["time"]) / self._search_time) * 100
+ except ZeroDivisionError:
+ query["width_ratio"] = 0
+
+ query["start_offset"] = width_ratio_tally
+ width_ratio_tally += query["width_ratio"]
+
+ context = self.context.copy()
+ context.update(
+ {
+ "backends": sorted(
+ self._backends.items(), key=lambda x: -x[1]["time_spent"]
+ ),
+ "queries": [q for a, q in self._queries],
+ "sql_time": self._search_time,
+ }
+ )
+
+~~ return render_to_string("panels/haystack.html", context)
+
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 7 from django-jet
+[django-jet](https://github.com/geex-arts/django-jet)
+([project documentation](https://jet.readthedocs.io/en/latest/),
+[PyPI project page](https://pypi.org/project/django-jet/) and
+[more information](http://jet.geex-arts.com/))
+is a fancy [Django](/django.html) Admin panel replacement.
+
+The django-jet project is open source under the
+[GNU Affero General Public License v3.0](https://github.com/geex-arts/django-jet/blob/dev/LICENSE).
+
+[**django-jet / jet / dashboard / dashboard.py**](https://github.com/geex-arts/django-jet/blob/dev/jet/dashboard/dashboard.py)
+
+```python
+# dashboard.py
+from importlib import import_module
+try:
+ from django.core.urlresolvers import reverse
+except ImportError: # Django 1.11
+ from django.urls import reverse
+
+~~from django.template.loader import render_to_string
+from jet.dashboard import modules
+from jet.dashboard.models import UserDashboardModule
+from django.utils.translation import ugettext_lazy as _
+from jet.ordered_set import OrderedSet
+from jet.utils import get_admin_site_name, context_to_dict
+
+try:
+ from django.template.context_processors import csrf
+except ImportError:
+ from django.core.context_processors import csrf
+
+
+class Dashboard(object):
+
+ columns = 2
+
+ children = None
+
+ available_children = None
+ app_label = None
+ context = None
+ modules = None
+
+ class Media:
+
+
+## ... source file abbreviated to get to render_to_string examples ...
+
+
+ user=self.context['request'].user.pk
+ ).all()
+
+ if len(module_models) == 0:
+ module_models = self.create_initial_module_models(self.context['request'].user)
+
+ loaded_modules = []
+
+ for module_model in module_models:
+ module_cls = module_model.load_module()
+ if module_cls is not None:
+ module = module_cls(model=module_model, context=self.context)
+ loaded_modules.append(module)
+
+ self.modules = loaded_modules
+
+ def render(self):
+ context = context_to_dict(self.context)
+ context.update({
+ 'columns': range(self.columns),
+ 'modules': self.modules,
+ 'app_label': self.app_label,
+ })
+ context.update(csrf(context['request']))
+
+~~ return render_to_string('jet.dashboard/dashboard.html', context)
+
+ def render_tools(self):
+ context = context_to_dict(self.context)
+ context.update({
+ 'children': self.children,
+ 'app_label': self.app_label,
+ 'available_children': self.available_children
+ })
+ context.update(csrf(context['request']))
+
+~~ return render_to_string('jet.dashboard/dashboard_tools.html', context)
+
+ def media(self):
+ unique_css = OrderedSet()
+ unique_js = OrderedSet()
+
+ for js in getattr(self.Media, 'js', ()):
+ unique_js.add(js)
+ for css in getattr(self.Media, 'css', ()):
+ unique_css.add(css)
+
+ for module in self.modules:
+ for js in getattr(module.Media, 'js', ()):
+ unique_js.add(js)
+ for css in getattr(module.Media, 'css', ()):
+ unique_css.add(css)
+
+ class Media:
+ css = list(unique_css)
+ js = list(unique_js)
+
+ return Media
+
+
+class AppIndexDashboard(Dashboard):
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 8 from django-pipeline
+[django-pipeline](https://github.com/jazzband/django-pipeline)
+([project documentation](https://django-pipeline.readthedocs.io/en/latest/)
+and
+[PyPI package information](https://pypi.org/project/django-pipeline/))
+is a code library for handling and compressing
+[static content assets](/static-content.html) when handling requests in
+[Django](/django.html) web applications.
+
+The django-pipeline project is open sourced under the
+[MIT License](https://github.com/jazzband/django-pipeline/blob/master/LICENSE.txt)
+and it is maintained by the developer community group
+[Jazzband](https://jazzband.co/).
+
+[**django-pipeline / pipeline / templatetags / pipeline.py**](https://github.com/jazzband/django-pipeline/blob/master/pipeline/templatetags/pipeline.py)
+
+```python
+# pipeline.py
+import logging
+import subprocess
+
+from django.contrib.staticfiles.storage import staticfiles_storage
+
+from django import template
+from django.template.base import VariableDoesNotExist
+~~from django.template.loader import render_to_string
+from django.utils.safestring import mark_safe
+
+from ..collector import default_collector
+from ..conf import settings
+from ..exceptions import CompilerError
+from ..packager import Packager, PackageNotFound
+from ..utils import guess_type
+
+logger = logging.getLogger(__name__)
+
+register = template.Library()
+
+
+class PipelineMixin(object):
+ request = None
+ _request_var = None
+
+ @property
+ def request_var(self):
+ if not self._request_var:
+ self._request_var = template.Variable('request')
+ return self._request_var
+
+ def package_for(self, package_name, package_type):
+
+
+## ... source file abbreviated to get to render_to_string examples ...
+
+
+ method = getattr(self, f'render_{package_type}')
+
+ return method(package, package.output_filename)
+
+ def render_compressed_sources(self, package, package_name, package_type):
+ if settings.PIPELINE_COLLECTOR_ENABLED:
+ default_collector.collect(self.request)
+
+ packager = Packager()
+ method = getattr(self, f'render_individual_{package_type}')
+
+ try:
+ paths = packager.compile(package.paths)
+ except CompilerError as e:
+ if settings.SHOW_ERRORS_INLINE:
+ method = getattr(self, f'render_error_{package_type}')
+ return method(package_name, e)
+ else:
+ raise
+
+ templates = packager.pack_templates(package)
+
+ return method(package, paths, templates=templates)
+
+ def render_error(self, package_type, package_name, e):
+~~ return render_to_string('pipeline/compile_error.html', {
+ 'package_type': package_type,
+ 'package_name': package_name,
+ 'command': subprocess.list2cmdline(e.command),
+ 'errors': e.error_output,
+ })
+
+
+class StylesheetNode(PipelineMixin, template.Node):
+ def __init__(self, name):
+ self.name = name
+
+ def render(self, context):
+ super(StylesheetNode, self).render(context)
+ package_name = template.Variable(self.name).resolve(context)
+
+ try:
+ package = self.package_for(package_name, 'css')
+ except PackageNotFound:
+ logger.warn("Package %r is unknown. Check PIPELINE['STYLESHEETS'] in your settings.", package_name)
+ return '' # fail silently, do not return anything if an invalid group is specified
+ return self.render_compressed(package, package_name, 'css')
+
+ def render_css(self, package, path):
+ template_name = package.template_name or "pipeline/css.html"
+ context = package.extra_context
+ context.update({
+ 'type': guess_type(path, 'text/css'),
+ 'url': mark_safe(staticfiles_storage.url(path))
+ })
+~~ return render_to_string(template_name, context)
+
+ def render_individual_css(self, package, paths, **kwargs):
+ tags = [self.render_css(package, path) for path in paths]
+ return '\n'.join(tags)
+
+ def render_error_css(self, package_name, e):
+ return super(StylesheetNode, self).render_error(
+ 'CSS', package_name, e)
+
+
+class JavascriptNode(PipelineMixin, template.Node):
+ def __init__(self, name):
+ self.name = name
+
+ def render(self, context):
+ super(JavascriptNode, self).render(context)
+ package_name = template.Variable(self.name).resolve(context)
+
+ try:
+ package = self.package_for(package_name, 'js')
+ except PackageNotFound:
+ logger.warn("Package %r is unknown. Check PIPELINE['JAVASCRIPT'] in your settings.", package_name)
+ return '' # fail silently, do not return anything if an invalid group is specified
+ return self.render_compressed(package, package_name, 'js')
+
+ def render_js(self, package, path):
+ template_name = package.template_name or "pipeline/js.html"
+ context = package.extra_context
+ context.update({
+ 'type': guess_type(path, 'text/javascript'),
+ 'url': mark_safe(staticfiles_storage.url(path))
+ })
+~~ return render_to_string(template_name, context)
+
+ def render_inline(self, package, js):
+ context = package.extra_context
+ context.update({
+ 'source': js
+ })
+~~ return render_to_string("pipeline/inline_js.html", context)
+
+ def render_individual_js(self, package, paths, templates=None):
+ tags = [self.render_js(package, js) for js in paths]
+ if templates:
+ tags.append(self.render_inline(package, templates))
+ return '\n'.join(tags)
+
+ def render_error_js(self, package_name, e):
+ return super(JavascriptNode, self).render_error(
+ 'JavaScript', package_name, e)
+
+
+@register.tag
+def stylesheet(parser, token):
+ try:
+ tag_name, name = token.split_contents()
+ except ValueError:
+ raise template.TemplateSyntaxError('%r requires exactly one argument: the name of a group in the PIPELINE.STYLESHEETS setting' % token.split_contents()[0])
+ return StylesheetNode(name)
+
+
+@register.tag
+def javascript(parser, token):
+ try:
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 9 from django-wiki
+[django-wiki](https://github.com/django-wiki/django-wiki)
+([project documentation](https://django-wiki.readthedocs.io/en/master/),
+[demo](https://demo.django-wiki.org/),
+and [PyPI page](https://pypi.org/project/django-wiki/))
+is a wiki system code library for [Django](/django.html)
+projects that makes it easier to create user-editable content.
+The project aims to provide necessary core features and then
+have an easy plugin format for additional features, rather than
+having every exhaustive feature built into the core system.
+django-wiki is a rewrite of an earlier now-defunct project
+named [django-simplewiki](https://code.google.com/p/django-simple-wiki/).
+
+The code for django-wiki is provided as open source under the
+[GNU General Public License 3.0](https://github.com/django-wiki/django-wiki/blob/master/COPYING).
+
+[**django-wiki / src/wiki / decorators.py**](https://github.com/django-wiki/django-wiki/blob/master/src/wiki/./decorators.py)
+
+```python
+# decorators.py
+from functools import wraps
+from urllib.parse import quote as urlquote
+
+from django.http import HttpResponseForbidden
+from django.http import HttpResponseNotFound
+from django.http import HttpResponseRedirect
+from django.shortcuts import get_object_or_404
+from django.shortcuts import redirect
+~~from django.template.loader import render_to_string
+from django.urls import reverse
+from wiki.conf import settings
+from wiki.core.exceptions import NoRootURL
+
+
+def response_forbidden(request, article, urlpath, read_denied=False):
+ if request.user.is_anonymous:
+ qs = request.META.get("QUERY_STRING", "")
+ if qs:
+ qs = urlquote("?" + qs)
+ else:
+ qs = ""
+ return redirect(settings.LOGIN_URL + "?next=" + request.path + qs)
+ else:
+ return HttpResponseForbidden(
+~~ render_to_string(
+ "wiki/permission_denied.html",
+ context={
+ "article": article,
+ "urlpath": urlpath,
+ "read_denied": read_denied,
+ },
+ request=request,
+ )
+ )
+
+
+def get_article( # noqa: max-complexity=23
+ func=None,
+ can_read=True,
+ can_write=False,
+ deleted_contents=False,
+ not_locked=False,
+ can_delete=False,
+ can_moderate=False,
+ can_create=False,
+):
+
+ def wrapper(request, *args, **kwargs):
+ from . import models
+
+ path = kwargs.pop("path", None)
+ article_id = kwargs.pop("article_id", None)
+
+ if path is not None:
+ try:
+ urlpath = models.URLPath.get_by_path(path, select_related=True)
+ except NoRootURL:
+ return redirect("wiki:root_create")
+ except models.URLPath.DoesNotExist:
+ try:
+ pathlist = list(
+ filter(
+ lambda x: x != "",
+ path.split("/"),
+ )
+ )
+ path = "/".join(pathlist[:-1])
+ parent = models.URLPath.get_by_path(path)
+ return HttpResponseRedirect(
+ reverse("wiki:create", kwargs={"path": parent.path})
+ + "?slug=%s" % pathlist[-1].lower()
+ )
+ except models.URLPath.DoesNotExist:
+ return HttpResponseNotFound(
+~~ render_to_string(
+ "wiki/error.html",
+ context={"error_type": "ancestors_missing"},
+ request=request,
+ )
+ )
+ if urlpath.article:
+ article = urlpath.article
+ else:
+ return_url = reverse("wiki:get", kwargs={"path": urlpath.parent.path})
+ urlpath.delete()
+ return HttpResponseRedirect(return_url)
+
+ elif article_id:
+ articles = models.Article.objects
+
+ article = get_object_or_404(articles, id=article_id)
+ try:
+ urlpath = models.URLPath.objects.get(articles__article=article)
+ except (
+ models.URLPath.DoesNotExist,
+ models.URLPath.MultipleObjectsReturned,
+ ):
+ urlpath = None
+
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
+
+## Example 10 from wagtail
+[wagtail](https://github.com/wagtail/wagtail)
+([project website](https://wagtail.io/)) is a fantastic
+[Django](/django.html)-based CMS with code that is open source
+under the
+[BSD 3-Clause "New" or "Revised" License](https://github.com/wagtail/wagtail/blob/master/LICENSE).
+
+[**wagtail / wagtail / snippets / widgets.py**](https://github.com/wagtail/wagtail/blob/master/wagtail/snippets/widgets.py)
+
+```python
+# widgets.py
+import json
+
+from django import forms
+from django.contrib.admin.utils import quote
+~~from django.template.loader import render_to_string
+from django.urls import reverse
+from django.utils.translation import gettext_lazy as _
+
+from wagtail.admin.staticfiles import versioned_static
+from wagtail.admin.widgets import AdminChooser
+from wagtail.admin.widgets.button import ListingButton
+
+
+class AdminSnippetChooser(AdminChooser):
+
+ def __init__(self, model, **kwargs):
+ self.target_model = model
+ name = self.target_model._meta.verbose_name
+ self.choose_one_text = _('Choose %s') % name
+ self.choose_another_text = _('Choose another %s') % name
+ self.link_to_chosen_text = _('Edit this %s') % name
+
+ super().__init__(**kwargs)
+
+ def get_value_data(self, value):
+ if value is None:
+ return None
+ elif isinstance(value, self.target_model):
+ instance = value
+ else: # assume instance ID
+ instance = self.target_model.objects.get(pk=value)
+
+ app_label = self.target_model._meta.app_label
+ model_name = self.target_model._meta.model_name
+ quoted_id = quote(instance.pk)
+ edit_url = reverse('wagtailsnippets:edit', args=[app_label, model_name, quoted_id])
+
+ return {
+ 'id': instance.pk,
+ 'string': str(instance),
+ 'edit_url': edit_url,
+ }
+
+ def render_html(self, name, value_data, attrs):
+ value_data = value_data or {}
+
+ original_field_html = super().render_html(name, value_data.get('id'), attrs)
+
+~~ return render_to_string("wagtailsnippets/widgets/snippet_chooser.html", {
+ 'widget': self,
+ 'original_field_html': original_field_html,
+ 'attrs': attrs,
+ 'value': bool(value_data), # only used by chooser.html to identify blank values
+ 'display_title': value_data.get('string', ''),
+ 'edit_url': value_data.get('edit_url', ''),
+ })
+
+ def render_js_init(self, id_, name, value_data):
+ model = self.target_model
+
+ return "createSnippetChooser({id}, {model});".format(
+ id=json.dumps(id_),
+ model=json.dumps('{app}/{model}'.format(
+ app=model._meta.app_label,
+ model=model._meta.model_name)))
+
+ @property
+ def media(self):
+ return forms.Media(js=[
+ versioned_static('wagtailsnippets/js/snippet-chooser-modal.js'),
+ versioned_static('wagtailsnippets/js/snippet-chooser.js'),
+ ])
+
+
+
+## ... source file continues with no further render_to_string examples...
+
+```
+
diff --git a/content/pages/examples/django/django-template-loader-select-template.markdown b/content/pages/examples/django/django-template-loader-select-template.markdown
new file mode 100644
index 000000000..1b4a15d0e
--- /dev/null
+++ b/content/pages/examples/django/django-template-loader-select-template.markdown
@@ -0,0 +1,124 @@
+title: django.template.loader select_template Example Code
+category: page
+slug: django-template-loader-select-template-examples
+sortorder: 500011394
+toc: False
+sidebartitle: django.template.loader select_template
+meta: Python example code that shows how to use the select_template callable from the django.template.loader module of the Django project.
+
+
+`select_template` is a callable within the `django.template.loader` module of the Django project.
+
+get_template
+and
+render_to_string
+are a couple of other callables within the `django.template.loader` package that also have code examples.
+
+## Example 1 from django-tables2
+[django-tables2](https://github.com/jieter/django-tables2)
+([projection documentation](https://django-tables2.readthedocs.io/en/latest/)
+and
+[PyPI page](https://pypi.org/project/django-tables2/))
+is a code library for [Django](/django.html) that simplifies creating and
+displaying tables in [Django templates](/django-templates.html),
+especially with more advanced features such as pagination and sorting.
+The project and its code are
+[available as open source](https://github.com/jieter/django-tables2/blob/master/LICENSE).
+
+[**django-tables2 / django_tables2 / templatetags / django_tables2.py**](https://github.com/jieter/django-tables2/blob/master/django_tables2/templatetags/django_tables2.py)
+
+```python
+# django_tables2.py
+import re
+from collections import OrderedDict
+
+from django import template
+from django.conf import settings
+from django.core.exceptions import ImproperlyConfigured
+from django.template import Node, TemplateSyntaxError
+~~from django.template.loader import get_template, select_template
+from django.templatetags.l10n import register as l10n_register
+from django.utils.html import escape
+from django.utils.http import urlencode
+
+import django_tables2 as tables
+from django_tables2.paginators import LazyPaginator
+from django_tables2.utils import AttributeDict
+
+register = template.Library()
+kwarg_re = re.compile(r"(?:(.+)=)?(.+)")
+context_processor_error_msg = (
+ "Tag {%% %s %%} requires django.template.context_processors.request to be "
+ "in the template configuration in "
+ "settings.TEMPLATES[]OPTIONS.context_processors) in order for the included "
+ "template tags to function correctly."
+)
+
+
+def token_kwargs(bits, parser):
+ if not bits:
+ return {}
+ kwargs = OrderedDict()
+ while bits:
+ match = kwarg_re.match(bits[0])
+
+
+## ... source file abbreviated to get to select_template examples ...
+
+
+ self.template_name = template_name
+
+ def render(self, context):
+ table = self.table.resolve(context)
+
+ request = context.get("request")
+
+ if isinstance(table, tables.Table):
+ pass
+ elif hasattr(table, "model"):
+ queryset = table
+
+ table = tables.table_factory(model=queryset.model)(queryset, request=request)
+ else:
+ klass = type(table).__name__
+ raise ValueError("Expected table or queryset, not {}".format(klass))
+
+ if self.template_name:
+ template_name = self.template_name.resolve(context)
+ else:
+ template_name = table.template_name
+
+ if isinstance(template_name, str):
+ template = get_template(template_name)
+ else:
+~~ template = select_template(template_name)
+
+ try:
+ table.context = context
+ table.before_render(request)
+
+ return template.render(context={"table": table}, request=request)
+ finally:
+ del table.context
+
+
+@register.tag
+def render_table(parser, token):
+ bits = token.split_contents()
+ bits.pop(0)
+
+ table = parser.compile_filter(bits.pop(0))
+ template = parser.compile_filter(bits.pop(0)) if bits else None
+
+ return RenderTableNode(table, template)
+
+
+register.filter("localize", l10n_register.filters["localize"])
+register.filter("unlocalize", l10n_register.filters["unlocalize"])
+
+
+
+## ... source file continues with no further select_template examples...
+
+```
+
diff --git a/content/pages/examples/django/django-template-loader-tags-blocknode.markdown b/content/pages/examples/django/django-template-loader-tags-blocknode.markdown
new file mode 100644
index 000000000..908fd9095
--- /dev/null
+++ b/content/pages/examples/django/django-template-loader-tags-blocknode.markdown
@@ -0,0 +1,192 @@
+title: django.template.loader_tags BlockNode Example Code
+category: page
+slug: django-template-loader-tags-blocknode-examples
+sortorder: 500011395
+toc: False
+sidebartitle: django.template.loader_tags BlockNode
+meta: Example code for understanding how to use the BlockNode class from the django.template.loader_tags module of the Django project.
+
+
+`BlockNode` is a class within the `django.template.loader_tags` module of the Django project.
+
+ExtendsNode
+and
+IncludeNode
+are a couple of other callables within the `django.template.loader_tags` package that also have code examples.
+
+## Example 1 from django-cms
+[django-cms](https://github.com/divio/django-cms)
+([project website](https://www.django-cms.org/en/)) is a Python-based
+content management system (CMS) [library](https://pypi.org/project/django-cms/)
+for use with Django web apps that is open sourced under the
+[BSD 3-Clause "New"](https://github.com/divio/django-cms/blob/develop/LICENSE)
+license.
+
+[**django-cms / cms / utils / placeholder.py**](https://github.com/divio/django-cms/blob/develop/cms/utils/placeholder.py)
+
+```python
+# placeholder.py
+import operator
+import warnings
+from collections import OrderedDict
+
+from django.conf import settings
+from django.core.exceptions import ImproperlyConfigured
+from django.db.models.query_utils import Q
+from django.template import TemplateSyntaxError, NodeList, Variable, Context, Template, engines
+from django.template.base import VariableNode
+from django.template.loader import get_template
+~~from django.template.loader_tags import BlockNode, ExtendsNode, IncludeNode
+
+from sekizai.helpers import get_varname
+
+from cms.exceptions import DuplicatePlaceholderWarning
+from cms.utils.conf import get_cms_setting
+
+
+def _get_nodelist(tpl):
+ if hasattr(tpl, 'template'):
+ return tpl.template.nodelist
+ else:
+ return tpl.nodelist
+
+
+def get_context():
+ if engines is not None:
+ context = Context()
+ context.template = Template('')
+ return context
+ else:
+ return {}
+
+
+def get_placeholder_conf(setting, placeholder, template=None, default=None):
+
+
+## ... source file abbreviated to get to BlockNode examples ...
+
+
+ validate_placeholder_name(slot)
+ placeholders.append(placeholder)
+ clean_placeholders.append(slot)
+ return placeholders
+
+
+def get_static_placeholders(template, context):
+ compiled_template = get_template(template)
+ nodes = _scan_static_placeholders(_get_nodelist(compiled_template))
+ placeholders = [node.get_declaration(context) for node in nodes]
+ placeholders_with_code = []
+
+ for placeholder in placeholders:
+ if placeholder.slot:
+ placeholders_with_code.append(placeholder)
+ else:
+ warnings.warn('Unable to resolve static placeholder '
+ 'name in template "{}"'.format(template),
+ Warning)
+ return placeholders_with_code
+
+
+def _get_block_nodes(extend_node):
+ parent = extend_node.get_parent(get_context())
+ parent_nodelist = _get_nodelist(parent)
+~~ parent_nodes = parent_nodelist.get_nodes_by_type(BlockNode)
+ parent_extend_nodes = parent_nodelist.get_nodes_by_type(ExtendsNode)
+
+ if parent_extend_nodes:
+ nodes = _get_block_nodes(parent_extend_nodes[0])
+ else:
+ nodes = OrderedDict()
+
+ for node in parent_nodes:
+ nodes[node.name] = node
+
+~~ current_nodes = _get_nodelist(extend_node).get_nodes_by_type(BlockNode)
+
+ for node in current_nodes:
+ if node.name in nodes:
+ node.super = nodes[node.name]
+ nodes[node.name] = node
+ return nodes
+
+
+def _get_placeholder_nodes_from_extend(extend_node, node_class):
+ block_nodes = _get_block_nodes(extend_node)
+ block_names = list(block_nodes.keys())
+
+ placeholders = []
+
+ for block in block_nodes.values():
+ placeholders.extend(_scan_placeholders(_get_nodelist(block), node_class, block, block_names))
+
+ parent_template = _find_topmost_template(extend_node)
+ placeholders += _scan_placeholders(_get_nodelist(parent_template), node_class, None, block_names)
+ return placeholders
+
+
+def _find_topmost_template(extend_node):
+ parent_template = extend_node.get_parent(get_context())
+ nodes.append(node)
+ elif isinstance(node, IncludeNode):
+ if node.template:
+ if not callable(getattr(node.template, 'render', None)):
+ if isinstance(node.template.var, Variable):
+ continue
+ else:
+ template = get_template(node.template.var)
+ else:
+ template = node.template
+ nodes += _scan_placeholders(_get_nodelist(template), node_class, current_block)
+ elif isinstance(node, ExtendsNode):
+ nodes += _get_placeholder_nodes_from_extend(node, node_class)
+ elif isinstance(node, VariableNode) and current_block:
+ if node.filter_expression.token == 'block.super':
+ if not hasattr(current_block.super, 'nodelist'):
+ raise TemplateSyntaxError("Cannot render block.super for blocks without a parent.")
+ nodes += _scan_placeholders(_get_nodelist(current_block.super), node_class, current_block.super)
+ elif isinstance(node, BlockNode) and node.name in ignore_blocks:
+ continue
+ elif hasattr(node, 'child_nodelists'):
+ for nodelist_name in node.child_nodelists:
+ if hasattr(node, nodelist_name):
+ subnodelist = getattr(node, nodelist_name)
+ if isinstance(subnodelist, NodeList):
+~~ if isinstance(node, BlockNode):
+ current_block = node
+ nodes += _scan_placeholders(subnodelist, node_class, current_block, ignore_blocks)
+ else:
+ for attr in dir(node):
+ obj = getattr(node, attr)
+ if isinstance(obj, NodeList):
+~~ if isinstance(node, BlockNode):
+ current_block = node
+ nodes += _scan_placeholders(obj, node_class, current_block, ignore_blocks)
+ return nodes
+
+
+def _scan_static_placeholders(nodelist):
+ from cms.templatetags.cms_tags import StaticPlaceholderNode
+
+ return _scan_placeholders(nodelist, node_class=StaticPlaceholderNode)
+
+
+def get_placeholders(template):
+ compiled_template = get_template(template)
+
+ placeholders = []
+ nodes = _scan_placeholders(_get_nodelist(compiled_template))
+ clean_placeholders = []
+
+ for node in nodes:
+ placeholder = node.get_declaration()
+ slot = placeholder.slot
+
+ if slot in clean_placeholders:
+ warnings.warn("Duplicate {{% placeholder \"{0}\" %}} "
+
+
+## ... source file continues with no further BlockNode examples...
+
+```
+
diff --git a/content/pages/examples/django/django-template-loader-tags-extendsnode.markdown b/content/pages/examples/django/django-template-loader-tags-extendsnode.markdown
new file mode 100644
index 000000000..61b2dd957
--- /dev/null
+++ b/content/pages/examples/django/django-template-loader-tags-extendsnode.markdown
@@ -0,0 +1,188 @@
+title: django.template.loader_tags ExtendsNode Example Code
+category: page
+slug: django-template-loader-tags-extendsnode-examples
+sortorder: 500011396
+toc: False
+sidebartitle: django.template.loader_tags ExtendsNode
+meta: Example code for understanding how to use the ExtendsNode class from the django.template.loader_tags module of the Django project.
+
+
+`ExtendsNode` is a class within the `django.template.loader_tags` module of the Django project.
+
+BlockNode
+and
+IncludeNode
+are a couple of other callables within the `django.template.loader_tags` package that also have code examples.
+
+## Example 1 from django-cms
+[django-cms](https://github.com/divio/django-cms)
+([project website](https://www.django-cms.org/en/)) is a Python-based
+content management system (CMS) [library](https://pypi.org/project/django-cms/)
+for use with Django web apps that is open sourced under the
+[BSD 3-Clause "New"](https://github.com/divio/django-cms/blob/develop/LICENSE)
+license.
+
+[**django-cms / cms / utils / placeholder.py**](https://github.com/divio/django-cms/blob/develop/cms/utils/placeholder.py)
+
+```python
+# placeholder.py
+import operator
+import warnings
+from collections import OrderedDict
+
+from django.conf import settings
+from django.core.exceptions import ImproperlyConfigured
+from django.db.models.query_utils import Q
+from django.template import TemplateSyntaxError, NodeList, Variable, Context, Template, engines
+from django.template.base import VariableNode
+from django.template.loader import get_template
+~~from django.template.loader_tags import BlockNode, ExtendsNode, IncludeNode
+
+from sekizai.helpers import get_varname
+
+from cms.exceptions import DuplicatePlaceholderWarning
+from cms.utils.conf import get_cms_setting
+
+
+def _get_nodelist(tpl):
+ if hasattr(tpl, 'template'):
+ return tpl.template.nodelist
+ else:
+ return tpl.nodelist
+
+
+def get_context():
+ if engines is not None:
+ context = Context()
+ context.template = Template('')
+ return context
+ else:
+ return {}
+
+
+def get_placeholder_conf(setting, placeholder, template=None, default=None):
+
+
+## ... source file abbreviated to get to ExtendsNode examples ...
+
+
+ placeholders.append(placeholder)
+ clean_placeholders.append(slot)
+ return placeholders
+
+
+def get_static_placeholders(template, context):
+ compiled_template = get_template(template)
+ nodes = _scan_static_placeholders(_get_nodelist(compiled_template))
+ placeholders = [node.get_declaration(context) for node in nodes]
+ placeholders_with_code = []
+
+ for placeholder in placeholders:
+ if placeholder.slot:
+ placeholders_with_code.append(placeholder)
+ else:
+ warnings.warn('Unable to resolve static placeholder '
+ 'name in template "{}"'.format(template),
+ Warning)
+ return placeholders_with_code
+
+
+def _get_block_nodes(extend_node):
+ parent = extend_node.get_parent(get_context())
+ parent_nodelist = _get_nodelist(parent)
+ parent_nodes = parent_nodelist.get_nodes_by_type(BlockNode)
+~~ parent_extend_nodes = parent_nodelist.get_nodes_by_type(ExtendsNode)
+
+ if parent_extend_nodes:
+ nodes = _get_block_nodes(parent_extend_nodes[0])
+ else:
+ nodes = OrderedDict()
+
+ for node in parent_nodes:
+ nodes[node.name] = node
+
+ current_nodes = _get_nodelist(extend_node).get_nodes_by_type(BlockNode)
+
+ for node in current_nodes:
+ if node.name in nodes:
+ node.super = nodes[node.name]
+ nodes[node.name] = node
+ return nodes
+
+
+def _get_placeholder_nodes_from_extend(extend_node, node_class):
+ block_nodes = _get_block_nodes(extend_node)
+ block_names = list(block_nodes.keys())
+
+ placeholders = []
+
+ for block in block_nodes.values():
+ placeholders.extend(_scan_placeholders(_get_nodelist(block), node_class, block, block_names))
+
+ parent_template = _find_topmost_template(extend_node)
+ placeholders += _scan_placeholders(_get_nodelist(parent_template), node_class, None, block_names)
+ return placeholders
+
+
+def _find_topmost_template(extend_node):
+ parent_template = extend_node.get_parent(get_context())
+~~ for node in _get_nodelist(parent_template).get_nodes_by_type(ExtendsNode):
+ return _find_topmost_template(node)
+ return extend_node.get_parent(get_context())
+
+
+def _scan_placeholders(nodelist, node_class=None, current_block=None, ignore_blocks=None):
+ from cms.templatetags.cms_tags import Placeholder
+
+ if not node_class:
+ node_class = Placeholder
+
+ nodes = []
+
+ if ignore_blocks is None:
+ ignore_blocks = []
+
+ for node in nodelist:
+ if isinstance(node, node_class):
+ nodes.append(node)
+ elif isinstance(node, IncludeNode):
+ if node.template:
+ if not callable(getattr(node.template, 'render', None)):
+ if isinstance(node.template.var, Variable):
+ continue
+ else:
+ template = get_template(node.template.var)
+ else:
+ template = node.template
+ nodes += _scan_placeholders(_get_nodelist(template), node_class, current_block)
+~~ elif isinstance(node, ExtendsNode):
+ nodes += _get_placeholder_nodes_from_extend(node, node_class)
+ elif isinstance(node, VariableNode) and current_block:
+ if node.filter_expression.token == 'block.super':
+ if not hasattr(current_block.super, 'nodelist'):
+ raise TemplateSyntaxError("Cannot render block.super for blocks without a parent.")
+ nodes += _scan_placeholders(_get_nodelist(current_block.super), node_class, current_block.super)
+ elif isinstance(node, BlockNode) and node.name in ignore_blocks:
+ continue
+ elif hasattr(node, 'child_nodelists'):
+ for nodelist_name in node.child_nodelists:
+ if hasattr(node, nodelist_name):
+ subnodelist = getattr(node, nodelist_name)
+ if isinstance(subnodelist, NodeList):
+ if isinstance(node, BlockNode):
+ current_block = node
+ nodes += _scan_placeholders(subnodelist, node_class, current_block, ignore_blocks)
+ else:
+ for attr in dir(node):
+ obj = getattr(node, attr)
+ if isinstance(obj, NodeList):
+ if isinstance(node, BlockNode):
+ current_block = node
+ nodes += _scan_placeholders(obj, node_class, current_block, ignore_blocks)
+ return nodes
+
+
+## ... source file continues with no further ExtendsNode examples...
+
+```
+
diff --git a/content/pages/examples/django/django-template-loader-tags-includenode.markdown b/content/pages/examples/django/django-template-loader-tags-includenode.markdown
new file mode 100644
index 000000000..412c41d09
--- /dev/null
+++ b/content/pages/examples/django/django-template-loader-tags-includenode.markdown
@@ -0,0 +1,124 @@
+title: django.template.loader_tags IncludeNode Example Code
+category: page
+slug: django-template-loader-tags-includenode-examples
+sortorder: 500011397
+toc: False
+sidebartitle: django.template.loader_tags IncludeNode
+meta: Example code for understanding how to use the IncludeNode class from the django.template.loader_tags module of the Django project.
+
+
+`IncludeNode` is a class within the `django.template.loader_tags` module of the Django project.
+
+BlockNode
+and
+ExtendsNode
+are a couple of other callables within the `django.template.loader_tags` package that also have code examples.
+
+## Example 1 from django-cms
+[django-cms](https://github.com/divio/django-cms)
+([project website](https://www.django-cms.org/en/)) is a Python-based
+content management system (CMS) [library](https://pypi.org/project/django-cms/)
+for use with Django web apps that is open sourced under the
+[BSD 3-Clause "New"](https://github.com/divio/django-cms/blob/develop/LICENSE)
+license.
+
+[**django-cms / cms / utils / placeholder.py**](https://github.com/divio/django-cms/blob/develop/cms/utils/placeholder.py)
+
+```python
+# placeholder.py
+import operator
+import warnings
+from collections import OrderedDict
+
+from django.conf import settings
+from django.core.exceptions import ImproperlyConfigured
+from django.db.models.query_utils import Q
+from django.template import TemplateSyntaxError, NodeList, Variable, Context, Template, engines
+from django.template.base import VariableNode
+from django.template.loader import get_template
+~~from django.template.loader_tags import BlockNode, ExtendsNode, IncludeNode
+
+from sekizai.helpers import get_varname
+
+from cms.exceptions import DuplicatePlaceholderWarning
+from cms.utils.conf import get_cms_setting
+
+
+def _get_nodelist(tpl):
+ if hasattr(tpl, 'template'):
+ return tpl.template.nodelist
+ else:
+ return tpl.nodelist
+
+
+def get_context():
+ if engines is not None:
+ context = Context()
+ context.template = Template('')
+ return context
+ else:
+ return {}
+
+
+def get_placeholder_conf(setting, placeholder, template=None, default=None):
+
+
+## ... source file abbreviated to get to IncludeNode examples ...
+
+
+
+
+def restore_sekizai_context(context, changes):
+ varname = get_varname()
+ sekizai_container = context.get(varname)
+ for key, values in changes.items():
+ sekizai_namespace = sekizai_container[key]
+ for value in values:
+ sekizai_namespace.append(value)
+
+
+def _scan_placeholders(nodelist, node_class=None, current_block=None, ignore_blocks=None):
+ from cms.templatetags.cms_tags import Placeholder
+
+ if not node_class:
+ node_class = Placeholder
+
+ nodes = []
+
+ if ignore_blocks is None:
+ ignore_blocks = []
+
+ for node in nodelist:
+ if isinstance(node, node_class):
+ nodes.append(node)
+~~ elif isinstance(node, IncludeNode):
+ if node.template:
+ if not callable(getattr(node.template, 'render', None)):
+ if isinstance(node.template.var, Variable):
+ continue
+ else:
+ template = get_template(node.template.var)
+ else:
+ template = node.template
+ nodes += _scan_placeholders(_get_nodelist(template), node_class, current_block)
+ elif isinstance(node, ExtendsNode):
+ nodes += _get_placeholder_nodes_from_extend(node, node_class)
+ elif isinstance(node, VariableNode) and current_block:
+ if node.filter_expression.token == 'block.super':
+ if not hasattr(current_block.super, 'nodelist'):
+ raise TemplateSyntaxError("Cannot render block.super for blocks without a parent.")
+ nodes += _scan_placeholders(_get_nodelist(current_block.super), node_class, current_block.super)
+ elif isinstance(node, BlockNode) and node.name in ignore_blocks:
+ continue
+ elif hasattr(node, 'child_nodelists'):
+ for nodelist_name in node.child_nodelists:
+ if hasattr(node, nodelist_name):
+ subnodelist = getattr(node, nodelist_name)
+ if isinstance(subnodelist, NodeList):
+ if isinstance(node, BlockNode):
+
+
+## ... source file continues with no further IncludeNode examples...
+
+```
+
diff --git a/content/pages/examples/django/django-template-loader.markdown b/content/pages/examples/django/django-template-loader.markdown
new file mode 100644
index 000000000..9098f4881
--- /dev/null
+++ b/content/pages/examples/django/django-template-loader.markdown
@@ -0,0 +1,1118 @@
+title: django.template loader Example Code
+category: page
+slug: django-template-loader-examples
+sortorder: 500011368
+toc: False
+sidebartitle: django.template loader
+meta: Python example code that shows how to use the loader callable from the django.template module of the Django project.
+
+
+`loader` is a callable within the `django.template` module of the Django project.
+
+Context,
+Engine,
+Library,
+Node,
+NodeList,
+Origin,
+RequestContext,
+Template,
+TemplateDoesNotExist,
+TemplateSyntaxError,
+Variable,
+context,
+engine,
+and library
+are several other callables with code examples from the same `django.template` package.
+
+## Example 1 from django-cms
+[django-cms](https://github.com/divio/django-cms)
+([project website](https://www.django-cms.org/en/)) is a Python-based
+content management system (CMS) [library](https://pypi.org/project/django-cms/)
+for use with Django web apps that is open sourced under the
+[BSD 3-Clause "New"](https://github.com/divio/django-cms/blob/develop/LICENSE)
+license.
+
+[**django-cms / cms / plugin_pool.py**](https://github.com/divio/django-cms/blob/develop/cms/./plugin_pool.py)
+
+```python
+# plugin_pool.py
+from operator import attrgetter
+
+from django.core.exceptions import ImproperlyConfigured
+from django.urls import re_path, include
+from django.template.defaultfilters import slugify
+from django.utils.encoding import force_text
+from django.utils.functional import cached_property
+from django.utils.module_loading import autodiscover_modules
+from django.utils.translation import get_language, deactivate_all, activate
+from django.template import TemplateDoesNotExist, TemplateSyntaxError
+
+from cms.exceptions import PluginAlreadyRegistered, PluginNotRegistered
+from cms.plugin_base import CMSPluginBase
+from cms.utils.conf import get_cms_setting
+from cms.utils.helpers import normalize_name
+
+
+class PluginPool:
+
+ def __init__(self):
+ self.plugins = {}
+ self.discovered = False
+
+ def _clear_cached(self):
+ if 'registered_plugins' in self.__dict__:
+ del self.__dict__['registered_plugins']
+
+ if 'plugins_with_extra_menu' in self.__dict__:
+ del self.__dict__['plugins_with_extra_menu']
+
+ if 'plugins_with_extra_placeholder_menu' in self.__dict__:
+ del self.__dict__['plugins_with_extra_placeholder_menu']
+
+ def discover_plugins(self):
+ if self.discovered:
+
+
+## ... source file abbreviated to get to loader examples ...
+
+
+ autodiscover_modules('cms_plugins')
+ self.discovered = True
+
+ def clear(self):
+ self.discovered = False
+ self.plugins = {}
+ self._clear_cached()
+
+ def validate_templates(self, plugin=None):
+ if plugin:
+ plugins = [plugin]
+ else:
+ plugins = self.plugins.values()
+ for plugin in plugins:
+ if (plugin.render_plugin and not type(plugin.render_plugin) == property
+ or hasattr(plugin.model, 'render_template')
+ or hasattr(plugin, 'get_render_template')):
+ if (plugin.render_template is None and
+ not hasattr(plugin, 'get_render_template')):
+ raise ImproperlyConfigured(
+ "CMS Plugins must define a render template, "
+ "a get_render_template method or "
+ "set render_plugin=False: %s" % plugin
+ )
+ elif not hasattr(plugin, 'get_render_template'):
+~~ from django.template import loader
+
+ template = plugin.render_template
+ if isinstance(template, str) and template:
+ try:
+~~ loader.get_template(template)
+ except TemplateDoesNotExist as e:
+ if str(e) == template:
+ raise ImproperlyConfigured(
+ "CMS Plugins must define a render template (%s) that exists: %s"
+ % (plugin, template)
+ )
+ else:
+ pass
+ except TemplateSyntaxError:
+ pass
+ else:
+ if plugin.allow_children:
+ raise ImproperlyConfigured(
+ "CMS Plugins can not define render_plugin=False and allow_children=True: %s"
+ % plugin
+ )
+
+ def register_plugin(self, plugin):
+ if not issubclass(plugin, CMSPluginBase):
+ raise ImproperlyConfigured(
+ "CMS Plugins must be subclasses of CMSPluginBase, %r is not."
+ % plugin
+ )
+ plugin_name = plugin.__name__
+
+
+## ... source file continues with no further loader examples...
+
+```
+
+
+## Example 2 from django-extensions
+[django-extensions](https://github.com/django-extensions/django-extensions)
+([project documentation](https://django-extensions.readthedocs.io/en/latest/)
+and [PyPI page](https://pypi.org/project/django-extensions/))
+is a [Django](/django.html) project that adds a bunch of additional
+useful commands to the `manage.py` interface. This
+[GoDjango video](https://www.youtube.com/watch?v=1F6G3ONhr4k) provides a
+quick overview of what you get when you install it into your Python
+environment.
+
+The django-extensions project is open sourced under the
+[MIT license](https://github.com/django-extensions/django-extensions/blob/master/LICENSE).
+
+[**django-extensions / django_extensions / management / modelviz.py**](https://github.com/django-extensions/django-extensions/blob/master/django_extensions/management/modelviz.py)
+
+```python
+# modelviz.py
+
+import datetime
+import os
+import re
+
+from django.apps import apps
+from django.db.models.fields.related import (
+ ForeignKey, ManyToManyField, OneToOneField, RelatedField,
+)
+from django.contrib.contenttypes.fields import GenericRelation
+~~from django.template import Context, Template, loader
+from django.utils.encoding import force_str
+from django.utils.safestring import mark_safe
+from django.utils.translation import activate as activate_language
+
+
+__version__ = "1.1"
+__license__ = "Python"
+__author__ = "Bas van Oostveen Conferences split into a couple of categories: community-run and vendor-run. For example, -WWDC is Apple's vendor-run +WWDC is Apple's vendor-run developer conference. They run the show and control the messaging. -PyCon US is the community-run +PyCon US is the community-run Python developer conference. There are a ton of vendors there as sponsors but no one company controls what happens at the conference.
diff --git a/content/posts/200308-financial-resources-developers.markdown b/content/posts/200308-financial-resources-developers.markdown
index fb0619334..80f2560d0 100644
--- a/content/posts/200308-financial-resources-developers.markdown
+++ b/content/posts/200308-financial-resources-developers.markdown
@@ -77,7 +77,7 @@ macroeconomic data trends.
is well-written, straightforward and accessible, particularly because
it clearly targets its software developer audience.
-* [Don't Quit Your Day Job](https://dqydj.com/) uses a ton of metrics
+* [Don't Quit Your Day Job](https://dqydj.com) uses a ton of metrics
and statistics to ground their articles on financial topics that
are often relevant specifically to software developers. For example,
the article on
diff --git a/content/posts/200630-report-errors-flask-web-apps-sentry.markdown b/content/posts/200630-report-errors-flask-web-apps-sentry.markdown
index e55c14c5c..2e9ec9684 100644
--- a/content/posts/200630-report-errors-flask-web-apps-sentry.markdown
+++ b/content/posts/200630-report-errors-flask-web-apps-sentry.markdown
@@ -260,5 +260,5 @@ or [@mattmakai](https://twitter.com/mattmakai). I am also on GitHub with
the username [mattmakai](https://github.com/mattmakai).
If you see an issue or error in this tutorial, please
-[fork the source repository on GitHub](https://github.com/mattmakai/fullstackpython/blob/master/content/posts/200630-report-errors-flask-web-apps-sentry.markdown)
+[fork the source repository on GitHub](https://github.com/mattmakai/fullstackpython.com/blob/master/content/posts/200630-report-errors-flask-web-apps-sentry.markdown)
and submit a pull request with the fix.
diff --git a/content/posts/200809-transcribe-recordings-speech-text-assemblyai.markdown b/content/posts/200809-transcribe-recordings-speech-text-assemblyai.markdown
index 61e5084b9..4a5027fbe 100644
--- a/content/posts/200809-transcribe-recordings-speech-text-assemblyai.markdown
+++ b/content/posts/200809-transcribe-recordings-speech-text-assemblyai.markdown
@@ -3,7 +3,7 @@ slug: transcribe-recordings-speech-text-assemblyai
meta: Learn to transcribe speech in recordings like MP3s into text with Python and AssemblyAI's API.
category: post
date: 2020-08-09
-modified: 2020-08-30
+modified: 2021-09-13
newsletter: False
headerimage: /img/headers/python-assemblyai.jpg
headeralt: Logos for the implementations used in this blog post. Copyright their respective owners.
@@ -137,7 +137,7 @@ Create a new directory named `pytranscribe` to store these files as
we write them. Then change into the new project directory.
```
-mkdir pytranscibe
+mkdir pytranscribe
cd pytranscribe
```
@@ -457,9 +457,7 @@ That's it, we've got our transcription!
You may be wondering what to do if the accuracy isn't where you need
it to be for your situation. That is where
[boosting accuracy for keywords or phrases](https://docs.assemblyai.com/guides/boosting-accuracy-for-keywords-or-phrases)
-and
-[selecting a model that better matches your data](https://docs.assemblyai.com/guides/transcribing-with-a-different-acoustic-or-custom-language-model)
-come in. You can use either of those two methods to boost the accuracy
+comes in. You can use either of those two methods to boost the accuracy
of your recordings to an acceptable level for your situation.
@@ -470,7 +468,7 @@ transcribe recordings with speech into text output.
Next, take a look at some of their more advanced documentation that goes
beyond the basics in this tutorial:
-* [Supported file formats](https://docs.assemblyai.com/overview/supported-file-formats)
+* [Supported file formats](https://docs.assemblyai.com/faqs/supported-file-formats)
* [Transcribing dual channel/stereo recordings](https://docs.assemblyai.com/guides/transcribing-dual-channel-stereo-recordings)
* [Getting speaker labels (speaker diarization)](https://docs.assemblyai.com/guides/getting-speaker-labels-speaker-diarization)
diff --git a/content/posts/201009-accurate-twilio-voice-call-recording-transcriptions-assemblyai.markdown b/content/posts/201009-accurate-twilio-voice-call-recording-transcriptions-assemblyai.markdown
index e178de998..a041498f9 100644
--- a/content/posts/201009-accurate-twilio-voice-call-recording-transcriptions-assemblyai.markdown
+++ b/content/posts/201009-accurate-twilio-voice-call-recording-transcriptions-assemblyai.markdown
@@ -14,7 +14,7 @@ is commonly used to initiate and receive phone calls, but the transcription
accuracy for [recordings](https://www.twilio.com/docs/voice/api/recording)
often leaves a lot to be desired. In this tutorial, we'll see how to connect an
outbound phone call powered by the Twilio Voice API with
-[AssemblyAI's deep learning transcription API](https://docs.assemblyai.com/guides/transcribing-an-audio-file-recording)
+[AssemblyAI's deep learning transcription API](https://docs.assemblyai.com/overview/getting-started)
to get significantly more accurate speech-to-text output.
@@ -546,7 +546,7 @@ that needs high quality text-to-speech transcription. If the results
aren't quite good enough for you, check out this tutorial on
[boosting accuracy for keywords or phrases](https://docs.assemblyai.com/guides/boosting-accuracy-for-keywords-or-phrases)
as well as
-[selecting a model that better matches your data](https://docs.assemblyai.com/guides/transcribing-with-a-different-acoustic-or-custom-language-model).
+[better matching your data with topic detection](https://docs.assemblyai.com/guides/iab-categorization).
## What now?
diff --git a/content/posts/210105-django-accurate-twilio-voice-transcriptions.markdown b/content/posts/210105-django-accurate-twilio-voice-transcriptions.markdown
new file mode 100644
index 000000000..c4b7d0018
--- /dev/null
+++ b/content/posts/210105-django-accurate-twilio-voice-transcriptions.markdown
@@ -0,0 +1,716 @@
+title: Using Django & AssemblyAI for More Accurate Twilio Call Transcriptions
+slug: django-accurate-twilio-voice-transcriptions
+meta: Use Python, Django and AssemblyAI's transcription API to improve recording accuracy for Twilio Programmable Voice phone calls.
+category: post
+date: 2021-01-05
+modified: 2021-09-13
+newsletter: False
+headerimage: /img/headers/django-assemblyai.jpg
+headeralt: Logos for the implementations used in this blog post. Copyright their respective owners.
+
+
+[Recording phone calls](https://www.twilio.com/docs/voice/tutorials/how-to-record-phone-calls-python)
+with one or more participants is easy with
+[Twilio's Programmable Voice API](https://www.twilio.com/docs/voice/quickstart/python),
+but the speech-to-text accuracy can be poor, especially for transcription
+of words from niche domains such as healthcare and engineering.
+[AssemblyAI's API for transcription](https://www.assemblyai.com/)
+provides much higher accuracy by default and through optional keyword lists.
+accuracy for [recordings](https://www.twilio.com/docs/voice/api/recording).
+
+In this tutorial, we'll record an outbound Twilio call recording to AssemblyAI's
+API to get significantly more accurate speech-to-text output.
+
+
+## Tutorial Prerequisites
+Ensure you have Python 3 installed, because Python 2 reached its
+end-of-life at the beginning of 2020 and is no longer supported.
+Preferrably, you should have
+[Python 3.7 or greater installed](https://www.python.org/downloads/)
+in your [development environment](/development-environments.html).
+This tutorial will also use:
+
+We will use the following dependencies to complete this
+tutorial:
+
+* [Django](/django.html) version 3.1.x, where *x* is the latest security
+ release
+* A [Twilio account](https://www.twilio.com/referral/w9pugq) and the
+ [Python Twilio helper library](https://pypi.org/project/twilio/)
+ version 6.45.2 or newer
+* [requests](https://requests.readthedocs.io/)
+ [version 2.24.0](https://pypi.org/project/requests/)
+* An [AssemblyAI](https://www.assemblyai.com/) account, which you can sign up for a [free key API access key here](https://app.assemblyai.com/login/)
+
+All code in this blog post is available open source under the MIT license
+on GitHub under the
+[django-accurate-twilio-voice-transcriptions directory of the blog-code-examples repository](https://github.com/fullstackpython/blog-code-examples).
+Use the source code as you desire for your own projects.
+
+
+## Configuring our development environment
+Change into the directory where you keep your Python
+[virtual environments](/virtual-environments-virtualenvs-venvs.html).
+Create a new virtualenv for this project using the following
+command.
+
+Start the Django project by creating a new
+[virtual environment](/virtual-environments-virtualenvs-venvs.html)
+using the following command. I recommend using a separate directory
+such as `~/venvs/` (the tilde is a shortcut for your user's `home`
+directory) so that you always know where all your virtualenvs are
+located.
+
+```bash
+python3 -m venv ~/venvs/djtranscribe
+```
+
+Activate the virtualenv with the `activate` shell script:
+
+```bash
+source ~/venvs/djtranscribe/bin/activate
+```
+
+After the above command is executed, the command prompt will
+change so that the name of the virtualenv is prepended to the
+original command prompt format, so if your prompt is just
+`$`, it will now look like the following:
+
+```bash
+(djtranscribe) $
+```
+
+Remember, you have to activate your virtualenv in every new terminal
+window where you want to use dependencies in the virtualenv.
+
+We can now install the [Django](https://pypi.org/project/Django/)
+package into the activated but otherwise empty virtualenv.
+
+```
+pip install django==3.1.3 requests==2.24.0 twilio==6.45.2
+```
+
+Look for output similar to the following to confirm the appropriate
+packages were installed correctly from PyPI.
+
+```
+(djtranscribe) $ pip install django==3.1.3 requests==2.24.0 twilio=6.45.2
+pip install django requests twilio
+Collecting django
+ Downloading Django-3.1.3-py3-none-any.whl (7.8 MB)
+ |████████████████████████████████| 7.8 MB 2.6 MB/s
+Collecting requests
+ Using cached requests-2.24.0-py2.py3-none-any.whl (61 kB)
+Collecting twilio
+ Downloading twilio-6.47.0.tar.gz (460 kB)
+ |████████████████████████████████| 460 kB 19.6 MB/s
+Collecting sqlparse>=0.2.2
+ Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
+ |████████████████████████████████| 42 kB 4.8 MB/s
+Collecting pytz
+ Downloading pytz-2020.4-py2.py3-none-any.whl (509 kB)
+ |████████████████████████████████| 509 kB 31.0 MB/s
+Collecting asgiref<4,>=3.2.10
+ Downloading asgiref-3.3.0-py3-none-any.whl (19 kB)
+Collecting chardet<4,>=3.0.2
+ Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
+Collecting idna<3,>=2.5
+ Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
+Collecting certifi>=2017.4.17
+ Using cached certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
+Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
+ Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
+ |████████████████████████████████| 127 kB 24.5 MB/s
+Collecting six
+ Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
+Collecting PyJWT>=1.4.2
+ Using cached PyJWT-1.7.1-py2.py3-none-any.whl (18 kB)
+Using legacy 'setup.py install' for twilio, since package 'wheel' is not installed.
+Installing collected packages: sqlparse, pytz, asgiref, django, chardet, idna, certifi, urllib3, requests, six, PyJWT, twilio
+ Running setup.py install for twilio ... done
+Successfully installed PyJWT-1.7.1 asgiref-3.3.0 certifi-2020.6.20 chardet-3.0.4 django-3.1.3 idna-2.10 pytz-2020.4 requests-2.24.0 six-1.15.0 sqlparse-0.4.1 twilio-6.47.0 urllib3-1.25.11
+
+```
+
+We can get started coding the application now that we have all of our
+required dependencies installed.
+
+
+## Starting our Django project
+Let's begin coding our application.
+
+We can use the [Django](/django.html) `django-admin` tool to create
+the boilerplate code structure to get our project started.
+Change into the directory where you develop your applications. For
+example, I typically use `/Users/matt/devel/py/` for all of my
+Python projects. Then run the following command to start a Django
+project named `djtranscribe`:
+
+```
+django-admin.py startproject djtranscribe
+```
+
+Note that in this tutorial we are using the same name for both the
+virtualenv and the Django project directory, but they can be
+different names if you prefer that for organizing your own projects.
+
+The `django-admin` command creates a directory named `djtranscribe`
+along with several subdirectories that you should be familiar with
+if you have previously worked with Django.
+
+Change directories into the new project.
+
+```
+cd djtranscribe
+```
+
+Create a new Django app within `djtranscribe` named `caller`.
+
+```
+python manage.py startapp caller
+```
+
+Django will generate a new folder named `caller` in the project.
+We should update the URLs so the app is accessible before we write
+our `views.py` code.
+
+Open `djtranscribe/djtranscribe/urls.py`. Add the highlighted
+lines so that URL resolver will check the `caller` app
+for additional routes to match with URLs that are requested of
+this Django application.
+
+```python
+# djtranscribe/djtranscribe/urls.py
+~~from django.conf.urls import include
+from django.contrib import admin
+from django.urls import path
+
+
+urlpatterns = [
+~~ path('', include('caller.urls')),
+ path('admin/', admin.site.urls),
+]
+```
+
+Save `djtranscribe/djtranscribe/urls.py` and open
+`djtranscribe/djtranscribe/settings.py`.
+Add the `caller` app to `settings.py` by inserting
+the highlighted line:
+
+```python
+# djtranscribe/djtranscribe/settings.py
+# Application definition
+
+INSTALLED_APPS = [
+ 'django.contrib.admin',
+ 'django.contrib.auth',
+ 'django.contrib.contenttypes',
+ 'django.contrib.sessions',
+ 'django.contrib.messages',
+ 'django.contrib.staticfiles',
+~~ 'caller',
+]
+```
+
+Make sure you change the default `DEBUG` and `SECRET_KEY`
+values in `settings.py` before you deploy any code to production. Secure
+your app properly with the information from the Django
+[production deployment checklist](https://docs.djangoproject.com/en/stable/howto/deployment/checklist/)
+so that you do not add your project to the list of hacked applications
+on the web.
+
+Save and close `settings.py`.
+
+Next change into the `djtranscribe/caller` directory. Create
+a new file named `urls.py` to contain routes for the `caller` app.
+
+Add all of these lines to the empty `djtranscribe/caller/urls.py`
+file.
+
+```python
+# djtranscribe/caller/urls.py
+from django.conf.urls import url
+from . import views
+
+urlpatterns = [
+ url(r'', views.index, name="index"),
+]
+```
+
+Save `djtranscribe/caller/urls.py`. Open
+`djtranscribe/caller/views.py` to add the
+following two highlighted lines.
+
+```
+# djtranscribe/caller/views.py
+from django.http import HttpResponse
+
+
+~~def index(request):
+~~ return HttpResponse('Hello, world!', 200)
+```
+
+We can test out that this simple boilerplate response is
+correct before we start adding the meat of the functionality to
+the project. Change into the base directory of your Django project
+where the `manage.py` file is located. Execute the development
+server with the following command:
+
+```bash
+python manage.py runserver
+```
+
+The Django development server should start up with no issues other than
+an unapplied migrations warning.
+
+```
+Watching for file changes with StatReloader
+Performing system checks...
+
+System check identified no issues (0 silenced).
+
+November 15, 2020 - 14:07:03
+Django version 3.1.3, using settings 'djtranscribe.settings'
+Starting development server at http://127.0.0.1:8000/
+Quit the server with CONTROL-C.
+```
+
+Open a web browser and go to `localhost:8000`.
+
+
+
+You should see 'Hello, world!' rendered in the browser.
+That means everything is working properly so far and we can
+now add the dialing, recording and transcribing capabilities to
+our Django project.
+
+
+## Adding Twilio to the Django project
+Time to add Twilio's Voice API into the mix so we can dial
+a phone call from our Django project and make a recording
+out of it.
+
+
+Start by opening up `djtranscribe/djtranscribe/settings.py`
+and modifying it with the following highlighted `import os`
+line:
+
+```python
+# djtranscribe/djtranscribe/settings.py
+~~import os
+from pathlib import Path
+
+
+# Build paths inside the project like this: BASE_DIR / 'subdir'.
+BASE_DIR = Path(__file__).resolve().parent.parent
+```
+
+Then at the bottom of the `settings.py` file, add the
+following highlighted lines, which will be settings that are pulled from
+environment variables we will configure later.
+
+```python
+# Static files (CSS, JavaScript, Images)
+# https://docs.djangoproject.com/en/3.1/howto/static-files/
+
+STATIC_URL = '/static/'
+
+
+~~BASE_URL = os.getenv("BASE_URL")
+~~TWIML_INSTRUCTIONS_URL = "{}/record/".format(BASE_URL)
+~~TWILIO_PHONE_NUMBER = os.getenv("TWILIO_PHONE_NUMBER")
+```
+
+Save `settings.py` and change into the `caller` Django app directory.
+
+Update `djtranscribe/caller/urls.py` with the the following new
+code:
+
+```python
+# djtranscribe/caller/urls.py
+from django.conf.urls import url
+from . import views
+
+urlpatterns = [
+~~ url(r'dial/(\d+)/$', views.dial, name="dial"),
+~~ url(r'record/$', views.record_twiml, name="record-twiml"),
+~~ url(r'get-recording-url/([A-Za-z0-9]+)/$', views.get_recording_url,
+~~ name='recording-url'),
+]
+```
+
+Next, open `djtranscribe/views.py` and update it with the following
+code, replacing what already exists within the file:
+
+```python
+# djtranscribe/caller/views.py
+from django.conf import settings
+from django.http import HttpResponse
+from django.views.decorators.csrf import csrf_exempt
+
+from twilio.rest import Client
+from twilio.twiml.voice_response import VoiceResponse
+
+
+def dial(request, phone_number):
+ """Dials an outbound phone call to the number in the URL. Just
+ as a heads up you will never want to leave a URL like this exposed
+ without authentication and further phone number format verification.
+ phone_number should be just the digits with the country code first,
+ for example 14155559812."""
+ # pulls credentials from environment variables
+ twilio_client = Client()
+ call = twilio_client.calls.create(
+ to='+{}'.format(phone_number),
+ from_=settings.TWILIO_PHONE_NUMBER,
+ url=settings.TWIML_INSTRUCTIONS_URL,
+ )
+ print(call.sid)
+ return HttpResponse("dialing +{}. call SID is: {}".format(
+ phone_number, call.sid))
+
+
+@csrf_exempt
+def record_twiml(request):
+ """Returns TwiML which prompts the caller to record a message"""
+ # Start our TwiML response
+ response = VoiceResponse()
+
+ # Use
+
+When you sign up you should have a phone number assigned to your account.
+You can use that or
+[purchase a new phone number](https://www.twilio.com/console/phone-numbers/search)
+to use.
+
+Set three environment variables with the names `TWILIO_ACCOUNT_SID`,
+`TWILIO_AUTH_TOKEN`, and `TWILIO_PHONE_NUMBER` using the `export` command
+in your terminal. Make sure to replace the values with your own Account SID,
+Auth Token and Twilio phone number.
+
+```bash
+export TWILIO_ACCOUNT_SID=xxxxxxxxxxxxx # found in twilio.com/console
+export TWILIO_AUTH_TOKEN=yyyyyyyyyyyyyy # found in twilio.com/console
+export TWILIO_PHONE_NUMBER=+17166382453 # replace with your Twilio number
+```
+
+Note that you must use the `export` command in every command line window
+that you want this key to be accessible. The scripts we are writing will
+not be able to access the Twilio APIs if you do not have the tokens exported
+in the environment where you are running the script.
+
+There is one more environment variable to set before we can run `app.py`.
+We need to use Ngrok as a localhost tunnel so that Twilio's webhook can
+send an HTTP POST request to our Django application running on
+our local development environment.
+
+Run Ngrok in a new terminal window, because you will need to keep it
+running while we run our other Python code:
+
+```bash
+./ngrok http 8000
+```
+
+
+
+Copy the HTTPS version of the "Forwarding" URL and set the `BASE_URL`
+environment variable value to it. For example, in this screenshot you
+would set `BASE_URL` to `https://7764c1810ad3.ngrok.io` using the
+following command:
+
+```bash
+export BASE_URL=https://7764c1810ad3.ngrok.io # use your ngrok URL, or domain. no trailing slash
+```
+
+We also need to update `djtranscribe/djtranscribe/settings.py`'s
+`ALLOWED_HOSTS` list to include the Ngrok Forwarding URL otherwise
+the [webhook](/webhooks.html) from Twilio asking for instructions
+on how to handle the phone call will fail. Open the `settings.py`
+file and update the `ALLOWED_HOSTS` with your Ngrok Forwarding
+hostname list the following:
+
+```
+# SECURITY WARNING: keep the secret key used in production secret!
+SECRET_KEY = os.getenv('SECRET_KEY', 'development key')
+
+# SECURITY WARNING: don't run with debug turned on in production!
+DEBUG = True
+
+~~ALLOWED_HOSTS = ['7764c1810ad3.ngrok.io','127.0.0.1','localhost']
+
+
+# Application definition
+
+INSTALLED_APPS = [
+ 'django.contrib.admin',
+ 'django.contrib.auth',
+ 'django.contrib.contenttypes',
+ 'django.contrib.sessions',
+ 'django.contrib.messages',
+ 'django.contrib.staticfiles',
+ 'caller',
+]
+```
+
+Okay, we can finally re-run our Django web app. Ensure Ngrok is still
+running in a different window, your virtualenv is active and that in this
+terminal you have your four environment variables set, then run the
+`runserver` command in the root project directory where `manage.py`
+is located:
+
+```bash
+python manage.py runserver
+```
+
+Let's make our phone ring by testing the application.
+
+
+## Testing Twilio Programmable Voice Recording
+We can test our application by going to localhost on port 8000.
+Go to this URL in your web browser, replacing the "14155551234"
+with the phone number you want to call, where the person on the
+line will be recorded: http://localhost:8000/dial/14155551234.
+
+That number should now receive a phone call from your Twilio
+number. Pick up, record a message that you want to use to test
+the transcription, and then hang up.
+
+If you get an error, make sure all of your environment variables
+are set. You can check the values by using the echo command like
+this:
+
+```bash
+echo $BASE_URL
+```
+
+When the call is over, copy the call SID show on the web page
+so that we can use it to look up where the recording audio
+file is stored.
+
+
+
+Go to "localhost:8000/get-recording-url/" with the call SID
+at the end. For example,
+"localhost:8000/get-recording-url/CAda3f2f49ff4e8ef2be6b726edb998c92".
+
+
+
+Copy the entire output except for the ".json" at the end, then paste
+it into the web browser's URL bar, prepended with "api.twilio.com".
+For example,
+"https://api.twilio.com/2010-04-01/Accounts/ACe3737affa0d2e17561ad44c9d190e70c/Recordings/RE3b42cf470bef829c3680ded961a09300".
+This will bring up the recording. Copy the entire URL and we will use it
+as input into the AssemblyAI service.
+
+
+## Transcribing with the AssemblyAI API
+We can now use the AssemblyAI API for speech-to-text transcription on
+the call recording that was just made.
+
+[Sign up for an AssemblyAI account](https://app.assemblyai.com/login/)
+and log in to the
+[AssemblyAI dashboard](https://app.assemblyai.com/dashboard/), then
+copy "Your API token" as shown in this screenshot:
+
+
+
+We need to export our AssemblyAI API key as an environment variable
+so that our Python application can use it to authenticate with their
+API. We also need to pass the publicly-accessible URL for the recording,
+so we'll set that as an environment variable as well.
+
+```bash
+# make sure to replace this URL with the one for your recording
+export ASSEMBLYAI_KEY=your-api-key-here
+export RECORDING_URL=https://api.twilio.com/2010-04-01/Accounts/ACe3737affa0d2e17561ad44c9d190e70c/Recordings/RE3b42cf470bef829c3680ded961a09300
+```
+
+Create a new file named `transcribe.py` and write the following code in it:
+
+```python
+import os
+import requests
+
+endpoint = "https://api.assemblyai.com/v2/transcript"
+
+json = {
+ "audio_url": os.getenv("RECORDING_URL")
+}
+
+headers = {
+ "authorization": os.getenv("ASSEMBLYAI_KEY"),
+ "content-type": "application/json"
+}
+
+response = requests.post(endpoint, json=json, headers=headers)
+
+print(response.json())
+```
+
+The above code calls the AssemblyAI transcription service using
+the secret key and passes it the URL with the file recording.
+The script prints out the JSON response from the service,
+which will contain a transcription ID that we'll use to access
+the results after they finish processing.
+
+Run the script using the `python` command:
+
+```bash
+python transcribe.py
+```
+
+You will get back some JSON as output, similar what you see here:
+
+```bash
+{'audio_end_at': None, 'acoustic_model': 'assemblyai_default', 'text': None, 'audio_url': 'https://api.twilio.com/2010-04-01/Accounts/ACe3737affa0d2e17561ad44c9d190e70c/Recordings/RE3b42cf470bef829c3680ded961a09300', 'speed_boost': False, 'language_model': 'assemblyai_default', 'redact_pii': False, 'confidence': None, 'webhook_status_code': None, 'id': 'zibe9vwmx-82ce-476c-85a7-e82c09c67daf', 'status': 'queued',
+'boost_param': None, 'words': None, 'format_text': True, 'webhook_url': None, 'punctuate': True, 'utterances': None, 'audio_duration': None, 'auto_highlights': False, 'word_boost': [], 'dual_channel': None, 'audio_start_from': None}
+```
+
+Find the value contained with the `id` field of the JSON response. We need
+that value to look up the final result of our transcription. Copy the
+transcription ID and set it as an environment variable to use as input by
+the final script:
+
+```
+# replace with what's found within `id` from the JSON response
+export TRANSCRIPTION_ID=aksd19vwmx-82ce-476c-85a7-e82c09c67daf
+```
+
+We just need a little more Python that looks up the result and we'll be all
+done.
+
+
+## Retrieve the AssemblyAI Transcription
+AssemblyAI will be busy transcribing the recording. Depending on the size of
+the file it can take anywhere from a few seconds to a few minutes for the
+job to complete. We can use the following code to see if the job is still
+in progress or it has completed. If the transcription is done it will print
+the results to the terminal.
+
+Create a new file named `print_transcription.py` with the following code:
+
+```python
+import os
+import requests
+
+endpoint = "https://api.assemblyai.com/v2/transcript/{}".format(os.getenv("TRANSCRIPTION_ID"))
+
+headers = {
+ "authorization": os.getenv("ASSEMBLYAI_KEY"),
+}
+
+response = requests.get(endpoint, headers=headers)
+
+print(response.json())
+print("\n\n")
+print(response.json()['text'])
+```
+
+The code above in `print_transcription.py` is very similar to the code
+in the previous `transcribe.py` source file. imports `os` (operating system)
+from the Python standard library, as we did in the previous two files,
+to obtain the `TRANSCRIPTION_ID` and `ASSEMBLYAI_KEY` environment variable
+values.
+
+The `endpoint` is the AssemblyAI API endpoint for retrieving
+transcriptions. We set the appropriate `authorization` header and
+make the API call using the `requests.get` function. We then print
+out the JSON response as well as just the text that was transcribed.
+
+Time to test out this third file. Execute the following command in
+the terminal:
+
+```bash
+python print_transcription.py
+```
+
+Your output will be different based on your recording but you should see a
+result in the terminal similar to the following:
+
+```bash
+{'audio_end_at': None, 'acoustic_model': 'assemblyai_default', 'auto_highlights_result': None, 'text': 'An object relational mapper is a code library that automates the transfer of data stored in a relational database tables into objects that are more commonly used in application. Code or MS provide a high level abstraction upon a relational database that allows the developer to write Python code. Instead of sequel to create read update and delete data and schemas in their database developers can use the programming language that they are comfortable with comfortable to work with the database instead of writing sequel statements or short procedures.', 'audio_url': 'https://api.twilio.com/2010-04-01/Accounts/ACe3737affa0d2e17561ad44c9d190e70c/Recordings/RE3b42cf470bef829c3680ded961a09300', 'speed_boost': False, 'language_model': 'assemblyai_default', 'id': 'zibe9vwmx-82ce-476c-85a7-e82c09c67daf', 'confidence': 0.931797752808989, 'webhook_status_code': None, 'status': 'completed', 'boost_param': None, 'redact_pii': False, 'words': [{'text': 'An', 'confidence': 1.0, 'end': 90, 'start': 0}, {'text': 'object', 'confidence': 0.94, 'end': 570, 'start': 210}, {'text': 'relational', 'confidence': 0.89, 'end': 1080, 'start': 510}, {'text': 'mapper', 'confidence': 0.97, 'end': 1380, 'start': 1020}, {'text': 'is', 'confidence': 0.88, 'end': 1560, 'start': 1350}, {'text': 'a', 'confidence': 0.99, 'end': 1620, 'start': 1500}, {'text': 'code', 'confidence': 0.93, 'end': 1920, 'start': 1620}, {'text': 'library', 'confidence': 0.94, 'end': 2250, 'start': 1860}, {'text': 'that', 'confidence': 0.99, 'end': 2490, 'start': 2220}, {'text': 'automates', 'confidence': 0.93, 'end': 2940, 'start': 2430}, {'text': 'the', 'confidence': 0.95, 'end': 3150, 'start': 2910}, {'text': 'transfer', 'confidence': 0.98, 'end': 3510, 'start': 3090}, {'text': 'of', 'confidence':
+0.99, 'end': 3660, 'start': 3480}, {'text': 'data', 'confidence': 0.84, 'end': 3960, 'start': 3630}, {'text': 'stored', 'confidence': 0.89, 'end': 4350, 'start': 3900}, {'text': 'in', 'confidence': 0.98, 'end': 4500, 'start': 4290}, {'text': 'a', 'confidence': 0.85, 'end': 4560, 'start': 4440}, {'text': 'relational', 'confidence': 0.87, 'end': 5580, 'start': 4500}, {'text': 'database', 'confidence': 0.92, 'end':
+6030, 'start': 5520}, {'text': 'tables', 'confidence': 0.93, 'end': 6330, 'start': 5970}, {'text': 'into', 'confidence': 0.92, 'end': 7130, 'start': 6560}, {'text': 'objects', 'confidence': 0.96, 'end': 7490, 'start': 7100}, {'text': 'that', 'confidence': 0.97, 'end': 7700, 'start': 7430}, {'text': 'are', 'confidence': 0.9, 'end': 7850, 'start': 7640}, {'text': 'more', 'confidence': 0.97, 'end': 8030, 'start': 7790}, {'text': 'commonly', 'confidence': 0.92, 'end': 8480, 'start': 7970}, {'text': 'used', 'confidence': 0.86, 'end': 8750, 'start': 8420}, {'text': 'in', 'confidence': 0.94, 'end': 9050, 'start': 8840}, {'text': 'application.', 'confidence': 0.98, 'end': 9860, 'start': 9110}, {'text': 'Code', 'confidence': 0.93, 'end': 10040, 'start': 9830}, {'text': 'or', 'confidence': 1.0, 'end': 11210, 'start': 10220}, {'text': 'MS', 'confidence': 0.83, 'end': 11480, 'start': 11180}, {'text': 'provide', 'confidence': 0.94, 'end': 11870, 'start': 11510}, {'text': 'a', 'confidence': 1.0, 'end': 11960, 'start': 11840}, {'text': 'high', 'confidence': 1.0, 'end': 12200, 'start': 11930}, {'text': 'level', 'confidence': 0.94, 'end': 12440, 'start': 12170}, {'text': 'abstraction', 'confidence': 0.95, 'end': 12980, 'start': 12410}, {'text':
+'upon', 'confidence': 0.94, 'end': 13220, 'start': 12950}, {'text': 'a', 'confidence': 1.0, 'end': 13280, 'start': 13160}, {'text': 'relational', 'confidence': 0.94, 'end': 13820, 'start': 13280}, {'text': 'database', 'confidence': 0.95, 'end': 14210, 'start': 13790}, {'text': 'that', 'confidence': 0.96, 'end': 14420, 'start': 14150}, {'text': 'allows', 'confidence': 0.99, 'end': 14720, 'start': 14360}, {'text':
+'the', 'confidence': 0.56, 'end': 14870, 'start': 14690}, {'text': 'developer', 'confidence': 0.98, 'end': 15290, 'start': 14810}, {'text': 'to', 'confidence': 0.94, 'end': 15410, 'start': 15230}, {'text': 'write', 'confidence': 0.96, 'end': 15680, 'start': 15380}, {'text': 'Python', 'confidence': 0.94, 'end': 16070, 'start': 15620}, {'text': 'code.', 'confidence': 0.98, 'end': 16310, 'start': 16070}, {'text': 'Instead', 'confidence': 0.97, 'end': 17160, 'start': 16500}, {'text': 'of', 'confidence': 0.93, 'end': 17340, 'start': 17130}, {'text': 'sequel', 'confidence': 0.86, 'end': 17820, 'start': 17280}, {'text': 'to', 'confidence': 0.91, 'end': 18090, 'start': 17880}, {'text': 'create', 'confidence': 0.89, 'end': 18450, 'start': 18090}, {'text': 'read', 'confidence': 0.88, 'end': 18840, 'start': 18480}, {'text': 'update', 'confidence': 0.92, 'end': 19290, 'start': 18870}, {'text': 'and', 'confidence': 0.94, 'end': 19590, 'start': 19230}, {'text': 'delete', 'confidence': 0.89, 'end': 19920, 'start': 19530}, {'text': 'data',
+'confidence': 0.95, 'end': 20190, 'start': 19890}, {'text': 'and', 'confidence': 0.92, 'end': 20490, 'start': 20250}, {'text': 'schemas', 'confidence': 0.86, 'end': 21000, 'start': 20430}, {'text': 'in', 'confidence': 0.94, 'end': 21210, 'start': 21000}, {'text': 'their', 'confidence': 0.98, 'end': 21510, 'start': 21150}, {'text': 'database', 'confidence': 0.97, 'end': 21900, 'start': 21450}, {'text': 'developers', 'confidence': 0.83, 'end': 23200, 'start': 22420}, {'text': 'can', 'confidence': 0.95, 'end': 23440, 'start': 23200}, {'text': 'use', 'confidence': 0.97, 'end': 23650, 'start': 23410}, {'text': 'the', 'confidence': 0.99, 'end': 23890, 'start': 23590}, {'text': 'programming', 'confidence': 0.97, 'end': 24370, 'start': 23830}, {'text': 'language', 'confidence': 1.0, 'end': 24700, 'start': 24310}, {'text': 'that', 'confidence': 1.0, 'end': 24880, 'start': 24640}, {'text': 'they', 'confidence': 0.99, 'end': 25060, 'start': 24820}, {'text': 'are', 'confidence': 0.85, 'end': 25210, 'start': 25000}, {'text': 'comfortable', 'confidence': 0.92, 'end': 25780, 'start': 25180}, {'text': 'with', 'confidence': 1.0, 'end': 25960, 'start': 25720}, {'text': 'comfortable', 'confidence': 0.94, 'end': 29090, 'start': 28090}, {'text': 'to', 'confidence': 0.84, 'end': 29840, 'start': 29180}, {'text': 'work', 'confidence': 0.95, 'end': 30050, 'start': 29780}, {'text': 'with', 'confidence': 0.98, 'end': 30290, 'start': 30020}, {'text': 'the', 'confidence': 0.69, 'end': 30440, 'start': 30230}, {'text': 'database', 'confidence': 0.98, 'end': 30860, 'start': 30380}, {'text': 'instead', 'confidence': 1.0, 'end': 32780, 'start': 31780}, {'text': 'of', 'confidence': 0.98, 'end': 32900, 'start': 32720}, {'text': 'writing', 'confidence': 0.87, 'end': 33320, 'start': 32870}, {'text': 'sequel', 'confidence': 0.88, 'end': 33860, 'start': 33290}, {'text': 'statements', 'confidence': 0.95, 'end': 34310, 'start': 33800}, {'text': 'or', 'confidence': 0.9, 'end': 34460, 'start': 34250}, {'text': 'short', 'confidence': 0.9, 'end': 34790, 'start': 34430}, {'text': 'procedures.', 'confidence': 0.98, 'end': 35270, 'start': 34760}], 'format_text': True, 'webhook_url': None, 'punctuate': True, 'utterances': None, 'audio_duration': 36.288, 'auto_highlights': False, 'word_boost': [],
+'dual_channel': None, 'audio_start_from': None}
+
+
+An object relational mapper is a code library that automates the transfer of data stored in a relational database tables into objects that are more commonly used in application. Code or MS provide a high level abstraction upon a relational database that allows the developer to write Python code. Instead of sequel to create read update and delete data and schemas in their database developers can use the programming language that they are comfortable with comfortable to work with the database instead of writing sequel statements or short procedures.
+```
+
+That's a lot of output. The first part contains the results of the
+transcription and the confidence in the accuracy of each word transcribed.
+The second part is just the plain text output from the transcription.
+
+You can take this now take this base code and add it to any application
+that needs high quality text-to-speech transcription. If the results
+aren't quite good enough for you yet, check out this tutorial on
+[boosting accuracy for keywords or phrases](https://docs.assemblyai.com/guides/boosting-accuracy-for-keywords-or-phrases).
+
+
+## Additional resources
+We just finished building a highly accurate transcription application for recordings.
+
+Next, try out some of these other related [Django](/django.html) tutorials:
+
+* [Using Sentry to Handle Python Exceptions in Django Projects](/blog/sentry-handle-exceptions-django-projects.html)
+* [Tracking Daily User Data in Django with django-user-visit](/blog/track-daily-user-data-django-user-visit.html)
+* [How to Quickly Use Bootstrap 4 in a Django Template with a CDN](/blog/bootstrap-4-django-template.html)
+
+Questions? Let me know via
+[a GitHub issue ticket on the Full Stack Python repository](https://github.com/mattmakai/fullstackpython.com/issues),
+on Twitter
+[@fullstackpython](https://twitter.com/fullstackpython)
+or [@mattmakai](https://twitter.com/mattmakai).
+See something wrong with this post? Fork
+[this page's source on GitHub](https://github.com/mattmakai/fullstackpython.com/blob/master/content/posts/210105-django-accurate-twilio-voice-transcriptions.markdown)
+and submit a pull request.
+
diff --git a/content/posts/210422-monitor-python-aws-lambda-sentry.markdown b/content/posts/210422-monitor-python-aws-lambda-sentry.markdown
new file mode 100644
index 000000000..00d3e4cba
--- /dev/null
+++ b/content/posts/210422-monitor-python-aws-lambda-sentry.markdown
@@ -0,0 +1,333 @@
+title: How to Monitor Python Functions on AWS Lambda with Sentry
+slug: monitor-python-functions-aws-lambda-sentry
+meta: Learn how to monitor your Python 3 functions on AWS Lambda using Sentry.
+category: post
+date: 2021-04-22
+modified: 2021-04-23
+newsletter: False
+headerimage: /img/headers/python-lambda-sentry.jpg
+headeralt: The Python, AWS Lambda and Sentry logos are copyright their respective owners.
+
+
+[Amazon Web Services (AWS) Lambda](/aws-lambda.html) is a usage-based
+compute service that can run [Python 3](/why-use-python.html) code. Errors
+can happen in any environment you are running your application in, so
+it is necessary to have reliable [monitoring](/monitoring.html) in place
+to have visibility when a problem occurs.
+
+In this post we will install and configure
+[Sentry](https://sentry.io/welcome/)'s application monitoring
+service that works specifically for code running on AWS Lambda.
+
+
+## Application Dependencies
+A local [development environment](/development-environments.html) is not
+required to follow this tutorial because all of the coding and configuration
+can happen in a web browser through the
+[AWS Console](https://console.aws.amazon.com/console/).
+
+The example code can be copy and pasted from this blog post or you
+can access it on GitHub under the
+[Full Stack Python blog-post-examples](https://github.com/fullstackpython/blog-code-examples)
+repository within the
+[monitor-python-aws-lambda-sentry directory](https://github.com/fullstackpython/blog-code-examples/tree/master/monitor-python-aws-lambda-sentry).
+
+
+## Accessing the AWS Lambda Service
+[Sign into your existing AWS account](https://aws.amazon.com/console)
+or sign up for a [new account](https://aws.amazon.com/). Lambda
+gives you the first 1 million requests for free so that you can execute
+basic applications without no or low cost.
+
+
+
+When you log into your account, use the search box to enter
+"lambda" and select "Lambda" when it appears to get to the right
+page.
+
+
+
+If you have already used Lambda before, you will see your existing Lambda
+functions in a searchable table. We're going to create a new function so
+click the "Create function" button.
+
+
+
+The create function page will give you several options for starting a new
+Lambda function.
+
+
+
+Click the "Browse Serverless App Repository" selection box, then choose
+the "hello-world-python3" starter app from within the
+"Public applications" section.
+
+
+
+The hello-world-python3 starter app details page should look something
+like the following screen:
+
+
+
+Fill in some example text such as "test" under `IdentityNameParameter`
+and click the "Deploy" button:
+
+
+
+The function will now be deployed. As soon as it is ready we can
+customize it and test it out before adding Sentry to capture any errors
+that occur during execution.
+
+
+## Testing the starter Python app
+Go back to the Lambda functions main page and select your new deployed
+starter app from the list.
+
+
+
+Find the orange "Test" button with a down arrow next to it like you
+see in the image below, and then click the down arrow. Select
+"Configure Test Event".
+
+
+
+Fill in the Event name as "FirstTest" or something similar, then
+press the "Create" button at the bottom of the modal window.
+
+Click the "Test" button and it will run the Lambda function with
+the parameters from that new test event. You should see something
+like the following output:
+
+```python
+Response
+"value1"
+
+Function Logs
+START RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914 Version: $LATEST
+value1 = value1
+value2 = value2
+value3 = value3
+END RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914
+REPORT RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914 Duration: 0.30 ms Billed Duration: 1 ms Memory Size: 128 MB Max Memory Used: 43 MB Init Duration: 1.34 ms
+
+Request ID
+62fa2f25-669c-47b7-b4e7-47353b0bd914
+```
+
+That means the test case was successful, but what happens even if there
+is a straightforward mistake in the code, such as trying to access an
+undeclared variable?
+
+Go into the code editor and you should see the starter code like this:
+
+
+
+Update the code with the new highlighted line, which tries to access
+a fourth variable, which does not exist in the test configuration
+we try to run it with.
+
+```python
+import json
+
+print('Loading function')
+
+
+def lambda_handler(event, context):
+ #print("Received event: " + json.dumps(event, indent=2))
+ print("value1 = " + event['key1'])
+ print("value2 = " + event['key2'])
+ print("value3 = " + event['key3'])
+~~ print("value4 = " + event['key4'])
+ return event['key1'] # Echo back the first key value
+ #raise Exception('Something went wrong')
+```
+
+After adding that one new line of code, hit the "Deploy" button,
+then the "Test" button. You should see some error output:
+
+```
+Response
+{
+ "errorMessage": "'key4'",
+ "errorType": "KeyError",
+ "stackTrace": [
+ [
+ "/var/task/lambda_function.py",
+ 11,
+ "lambda_handler",
+ "print(\"value4 = \" + event['key4'])"
+ ]
+ ]
+}
+
+Function Logs
+START RequestId: a4e956bd-cce4-403e-b5e7-e95bc3ffa2cb Version: $LATEST
+value1 = value1
+value2 = value2
+value3 = value3
+'key4': KeyError
+Traceback (most recent call last):
+ File "/var/task/lambda_function.py", line 11, in lambda_handler
+ print("value4 = " + event['key4'])
+KeyError: 'key4'
+
+END RequestId: a4e956bd-cce4-403e-b5e7-e95bc3ffa2cb
+REPORT RequestId: a4e956bd-cce4-403e-b5e7-e95bc3ffa2cb Duration: 0.81 ms Billed Duration: 1 ms Memory Size: 128 MB Max Memory Used: 43 MB Init Duration: 1.61 ms
+
+Request ID
+a4e956bd-cce4-403e-b5e7-e95bc3ffa2cb
+```
+
+It is obvious when we are working in the Console that an error just
+occurred. However, in most cases an error will happen sporadically
+which is why we need a monitoring system in place to catch and report
+on those exceptions.
+
+
+## AWS Lambda function monitoring with Sentry
+The easiest way to add Sentry to Lambda for this application
+is to configure an
+[AWS Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html)
+with the necessary dependency for Sentry. Sentry has concise
+[documentation on addin gvia Lambda Layers](https://docs.sentry.io/platforms/python/guides/aws-lambda/layer/)
+so we will walk through that way to configure it and test it
+out.
+
+First, scroll down to the "Layers" section while in your Lambda
+function configuration. Click the "Add a layer" button":
+
+
+
+In the "Add layer" screen, select the "Specify an ARN" option.
+
+
+
+Now to specify the Amazon Resource Name (ARN), we need to use
+the Sentry documentation to get the right configuration string.
+
+US-East-1 is the oldest and most commonly-used region so I'll
+use that here in this tutorial but you should check which one
+you are in if you are not certain.
+
+
+
+Copy that value into the Lambda Layer configuration, like this:
+
+
+
+Then press the "Add" button. Now you have the Sentry dependency
+in your environment so code that relies upon that library can be
+used in the Lambda function.
+
+Next we need to go into the Sentry dashboard to create a project,
+get our unique identifer, and connect it to our Lambda function.
+
+Sentry can be [self-hosted](https://github.com/getsentry/onpremise) or
+used as a cloud service through [Sentry.io](https://sentry.io). We will
+use the cloud hosted version because it is quicker than
+setting up your own server as well as free for smaller projects.
+
+Go to [Sentry.io's homepage](https://sentry.io).
+
+
+
+Sign into your account or sign up for a new free account. You will be at
+the main account dashboard after logging in or completing the Sentry sign
+up process.
+
+There are no errors logged on our account dashboard yet, which is as
+expected because we have not yet connected our account to our Lambda
+function.
+
+Click "Projects" on the left navigation bar, then "Create Project"
+in the top right corner.
+
+Under "Choose a Platform", select "Serverless" and then "AWS Lambda (Python)"
+as shown below:
+
+
+
+Decide under what criteria it should send error information out of
+Lambda. For this tutorial, we will have it send every exception.
+Then click the "Create Project." button.
+
+You can have Sentry handle the instrumentation automatically but
+we will handle it manually for our function. On the next screen, Sentry
+will provide you with your unique DSN string, which we will need for
+our function.
+
+
+
+Typically you will want to
+[use environment variables on AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html)
+to store and access values like your Sentry key.
+
+Copy the contents of the Sentry DSN string, and go into the Lambda console
+to create a new environment variable. To do that, click the "Configuration"
+tab within Lambda like you see here:
+
+
+
+Then click "Edit" and add a new environment variable with the key of `SENTRY_DSN`
+and the value of the DSN string that you copied from the Sentry screen.
+
+
+
+Click the "Save" button and go back to your Lambda function code.
+
+Update your Lambda function with the following highlighted new lines of code
+to send errors to Sentry.
+
+```python
+import json
+~~import os
+~~import sentry_sdk
+~~from sentry_sdk.integrations.aws_lambda import AwsLambdaIntegration
+
+~~SENTRY_DSN = os.environ.get('SENTRY_DSN')
+~~sentry_sdk.init(
+~~ dsn=SENTRY_DSN,
+~~ integrations=[AwsLambdaIntegration()]
+~~)
+
+print('Loading function')
+
+
+def lambda_handler(event, context):
+ #print("Received event: " + json.dumps(event, indent=2))
+ print("value1 = " + event['key1'])
+ print("value2 = " + event['key2'])
+ print("value3 = " + event['key3'])
+ print("value4 = " + event['key4'])
+ return event['key1'] # Echo back the first key value
+ #raise Exception('Something went wrong')
+```
+
+Click the "Deploy" button and then "Test". The code will throw
+an error and when we go back to our Sentry dashboard we will
+see it captured and viewable for further inspection.
+
+
+
+It works! Next you will likely want to tune your exception reporting
+criteria to make sure you get alerted to the right number of exceptions
+if you do not want to see all of them.
+
+
+## What's Next?
+We just wrote and executed a Python 3 function on AWS Lambda then
+captured the exception message into the Sentry logs. You can
+now continue building out your Python code knowing that when something
+goes wrong you will have full visibility on what happened.
+
+Check out the [AWS Lambda section](/aws-lambda.html) for
+more tutorials by other developers.
+
+Further questions? Contact me on Twitter
+[@fullstackpython](https://twitter.com/fullstackpython)
+or [@mattmakai](https://twitter.com/mattmakai). I am also on GitHub with
+the username [mattmakai](https://github.com/mattmakai).
+
+Something wrong with this post? Fork
+[this page's source on GitHub](https://github.com/mattmakai/fullstackpython.com/blob/master/content/posts/210422-monitor-python-aws-lambda-sentry.markdown)
+and submit a pull request.
diff --git a/content/posts/210823-performance-monitoring-aws-lambda-sentry.markdown b/content/posts/210823-performance-monitoring-aws-lambda-sentry.markdown
new file mode 100644
index 000000000..9ede782ef
--- /dev/null
+++ b/content/posts/210823-performance-monitoring-aws-lambda-sentry.markdown
@@ -0,0 +1,261 @@
+title: Application Performance Monitoring AWS Lambda Functions with Sentry
+slug: application-performance-monitoring-aws-lambda-functions-sentry
+meta: Learn how to use Sentry Application Performance Monitoring on AWS Lambda.
+category: post
+date: 2021-08-23
+modified: 2021-08-26
+newsletter: False
+headerimage: /img/headers/python-lambda-sentry.jpg
+headeralt: The Python, AWS Lambda and Sentry logos are copyright their respective owners.
+
+
+[Amazon Web Services (AWS) Lambda](/aws-lambda.html) is a usage-based
+computing infrastructure service that can execute
+[Python 3](/why-use-python.html) code. One of the challenges of this
+environment is ensuring efficient performance of your Lambda Functions.
+Application performance monitoring (APM) is particularly useful in these
+situations because you are billed based on how long you use the
+resources.
+
+In this post we will install and configure
+[Sentry's APM](https://sentry.io/for/performance/) that works via a
+[Lambda layer](https://docs.aws.amazon.com/lambda/latest/dg/invocation-layers.html).
+Note that if you are looking for error monitoring rather than performance
+monitoring, take a look at
+[How to Monitor Python Functions on AWS Lambda with Sentry](/blog/monitor-python-functions-aws-lambda-sentry.html)
+rather than following this post.
+
+
+## First steps with AWS Lambda
+A local [development environment](/development-environments.html) is not
+required to follow this tutorial because all of the coding and configuration
+can happen in a web browser through the
+[AWS Console](https://console.aws.amazon.com/console/).
+
+[Sign into your existing AWS account](https://aws.amazon.com/console)
+or sign up for a [new account](https://aws.amazon.com/). Lambda
+gives you the first 1 million requests for free so that you can execute
+basic applications without no or low cost.
+
+
+
+When you log into your account, use the search box to enter
+"lambda" and select "Lambda" when it appears to get to the right
+page.
+
+
+
+If you have already used Lambda before, you will see your existing Lambda
+functions in a searchable table. We're going to create a new function so
+click the "Create function" button.
+
+
+
+The create function page will give you several options for building a
+Lambda function.
+
+
+
+Click the "Browse Serverless App Repository" selection box, then choose
+the "hello-world-python3" starter app from within the
+"Public applications" section.
+
+
+
+The hello-world-python3 starter app details page should look something
+like the following screen:
+
+
+
+Fill in some example text such as "test" under `IdentityNameParameter`
+and click the "Deploy" button:
+
+
+
+The function will now be deployed. As soon as it is ready we can
+customize it and test it out before adding Sentry to capture any errors
+that occur during execution.
+
+Go back to the Lambda functions main page and select your new deployed
+starter app from the list.
+
+
+
+Find the orange "Test" button with a down arrow next to it like you
+see in the image below, and then click the down arrow. Select
+"Configure Test Event".
+
+
+
+Fill in the Event name as "FirstTest" or something similar, then
+press the "Create" button at the bottom of the modal window.
+
+Click the "Test" button and it will run the Lambda function with
+the parameters from that new test event. You should see something
+like the following output:
+
+```python
+Response
+"value1"
+
+Function Logs
+START RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914 Version: $LATEST
+value1 = value1
+value2 = value2
+value3 = value3
+END RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914
+REPORT RequestId: 62fa2f25-669c-47b7-b4e7-47353b0bd914 Duration: 0.30 ms Billed Duration: 1 ms Memory Size: 128 MB Max Memory Used: 43 MB Init Duration: 1.34 ms
+
+Request ID
+62fa2f25-669c-47b7-b4e7-47353b0bd914
+```
+
+The code was successfully executed, so let's add Sentry's performance
+monitoring and test some code that uses it.
+
+
+## Performance monitoring with Sentry
+Go to [Sentry.io's homepage](https://sentry.io).
+
+
+
+Sign into your account or sign up for a new free account. You will be at
+the main account dashboard after logging in or completing the Sentry sign
+up process.
+
+Select "Performance" on the left navigation bar, it will take you to the
+performance monitoring page.
+
+
+
+Click "Start Setup" then go back over to AWS Lambda to complete the
+steps for adding Sentry's Python layer to your Lambda function.
+
+The easiest way to add Sentry to Lambda for this application
+is to configure an
+[AWS Lambda Layer](https://docs.aws.amazon.com/lambda/latest/dg/configuration-layers.html)
+with the necessary dependency for Sentry. Sentry has concise
+[documentation on adding via Lambda Layers](https://docs.sentry.io/platforms/python/guides/aws-lambda/layer/)
+so we will walk through that way to configure it and test it
+out.
+
+Scroll down to the "Layers" section while in your Lambda
+function configuration. Click the "Add a layer" button":
+
+
+
+In the "Add layer" screen, select the "Specify an ARN" option.
+
+
+
+Now to specify the Amazon Resource Name (ARN), we need to use
+the Sentry documentation to get the right configuration string.
+
+US-East-1 is the oldest and most commonly-used region so I'll
+use that here in this tutorial but you should check which one
+you are in if you are not certain.
+
+
+
+Copy that value into the Lambda Layer configuration, like this:
+
+
+
+Then press the "Add" button. You now have the Sentry dependency
+in your environment so code that relies upon that library can be
+used in the Lambda function.
+
+
+## Testing performance monitoring
+Let's change our Python code in the Lambda function and test out
+the APM agent.
+
+Make sure you are signed into your Sentry account and go to
+[this specific AWS Lambda set up guide](https://docs.sentry.io/platforms/python/guides/aws-lambda/).
+
+You will see a "DSN string" that we need to set as an environment
+variable on AWS Lambda to finish our setup. Copy the string that
+matches your project as shown on that page in the highlighted green
+section:
+
+
+
+We will
+[use environment variables on AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html)
+to store and access values like this Sentry DSN key.
+
+Go into the Lambda console to create a new environment variable. To do
+that, click the "Configuration" tab within Lambda like you see here:
+
+
+
+Then click "Edit" and add a new environment variable with the key of `SENTRY_DSN`
+and the value of the DSN string that you copied from the Sentry screen.
+
+
+
+Click the "Save" button and go back to your Lambda function's code editor.
+
+Replace the code in your Lambda function with the following code:
+
+```python
+import json
+import os
+import sentry_sdk
+import time
+from sentry_sdk.integrations.aws_lambda import AwsLambdaIntegration
+from sentry_sdk import start_transaction
+
+SENTRY_DSN = os.environ.get('SENTRY_DSN')
+sentry_sdk.init(
+ dsn=SENTRY_DSN,
+ traces_sample_rate=1.0,
+ integrations=[AwsLambdaIntegration()]
+)
+
+print('Loading function')
+
+
+def lambda_handler(event, context):
+ calc = 1000
+
+ # this is custom instrumentation, see docs: https://bit.ly/2WjT3AY
+ with start_transaction(op="task", name="big calculation"):
+ for i in range(1, 1000):
+ calc = calc * i
+
+ print(calc)
+ return event['key1'] # Echo back the first key value
+```
+
+The above code imports the Sentry dependencies, and then runs both
+[automatic instrumentation](https://docs.sentry.io/platforms/python/guides/aws-lambda/performance/instrumentation/automatic-instrumentation/)
+and [custom instrumentation](https://bit.ly/2WjT3AY) on the
+code. Click the "Deploy" button and then "Test". The code will
+successfully execute and when we go back to our Sentry performance
+monitoring dashboard we will see some initial results, like this
+following screenshot.
+
+
+
+Looks good, you have both the default and the specified transaction
+performance recordings in the dashboard, and you can toggle between
+them (or other transactions you record) through the user interface.
+
+
+## What's Next?
+We just wrote and executed a Python 3 function on AWS Lambda that
+used the basics of Sentry APM to get some initial performance
+monitoring data.
+
+Check out the [AWS Lambda section](/aws-lambda.html) for
+more tutorials by other developers.
+
+Further questions? Contact me on Twitter
+[@fullstackpython](https://twitter.com/fullstackpython)
+or [@mattmakai](https://twitter.com/mattmakai). I am also on GitHub with
+the username [mattmakai](https://github.com/mattmakai).
+
+Something wrong with this post? Fork
+[this page's source on GitHub](https://github.com/mattmakai/fullstackpython.com/blob/master/content/posts/210823-performance-monitoring-aws-lambda-sentry.markdown)
+and submit a pull request.
diff --git a/static/img/210105-django-assemblyai/assemblyai-dashboard.png b/static/img/210105-django-assemblyai/assemblyai-dashboard.png
new file mode 100644
index 000000000..509b1af75
Binary files /dev/null and b/static/img/210105-django-assemblyai/assemblyai-dashboard.png differ
diff --git a/static/img/210105-django-assemblyai/call-recording-url.png b/static/img/210105-django-assemblyai/call-recording-url.png
new file mode 100644
index 000000000..fdf8d4bc4
Binary files /dev/null and b/static/img/210105-django-assemblyai/call-recording-url.png differ
diff --git a/static/img/210105-django-assemblyai/dial-call-sid.png b/static/img/210105-django-assemblyai/dial-call-sid.png
new file mode 100644
index 000000000..a79dacfb9
Binary files /dev/null and b/static/img/210105-django-assemblyai/dial-call-sid.png differ
diff --git a/static/img/210105-django-assemblyai/ngrok.jpg b/static/img/210105-django-assemblyai/ngrok.jpg
new file mode 100644
index 000000000..a56e5f21b
Binary files /dev/null and b/static/img/210105-django-assemblyai/ngrok.jpg differ
diff --git a/static/img/210105-django-assemblyai/twilio-console.png b/static/img/210105-django-assemblyai/twilio-console.png
new file mode 100644
index 000000000..7f0a3a66a
Binary files /dev/null and b/static/img/210105-django-assemblyai/twilio-console.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/add-env-var.jpg b/static/img/210406-python-sentry-aws-lambda/add-env-var.jpg
new file mode 100644
index 000000000..e1f45463d
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/add-env-var.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/add-lambda-layer.png b/static/img/210406-python-sentry-aws-lambda/add-lambda-layer.png
new file mode 100644
index 000000000..a5eb7c671
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/add-lambda-layer.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/add-layer-specify-arn.jpg b/static/img/210406-python-sentry-aws-lambda/add-layer-specify-arn.jpg
new file mode 100644
index 000000000..b97942149
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/add-layer-specify-arn.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/arn-region.png b/static/img/210406-python-sentry-aws-lambda/arn-region.png
new file mode 100644
index 000000000..c44697a65
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/arn-region.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/aws-lambda-configuration.jpg b/static/img/210406-python-sentry-aws-lambda/aws-lambda-configuration.jpg
new file mode 100644
index 000000000..4f5bb0e05
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/aws-lambda-configuration.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/aws-lambda-landing.jpg b/static/img/210406-python-sentry-aws-lambda/aws-lambda-landing.jpg
new file mode 100644
index 000000000..799013c14
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/aws-lambda-landing.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/aws-lambda-python.jpg b/static/img/210406-python-sentry-aws-lambda/aws-lambda-python.jpg
new file mode 100644
index 000000000..e3ba26649
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/aws-lambda-python.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/configure-test.jpg b/static/img/210406-python-sentry-aws-lambda/configure-test.jpg
new file mode 100644
index 000000000..0896d8e30
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/configure-test.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/create-function-detail.png b/static/img/210406-python-sentry-aws-lambda/create-function-detail.png
new file mode 100644
index 000000000..7e4c77748
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/create-function-detail.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/create-function.png b/static/img/210406-python-sentry-aws-lambda/create-function.png
new file mode 100644
index 000000000..33706be60
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/create-function.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/deploy-starter-app.png b/static/img/210406-python-sentry-aws-lambda/deploy-starter-app.png
new file mode 100644
index 000000000..4abdfe380
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/deploy-starter-app.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/functions-list.jpg b/static/img/210406-python-sentry-aws-lambda/functions-list.jpg
new file mode 100644
index 000000000..9229b338e
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/functions-list.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/hello-world-python3.png b/static/img/210406-python-sentry-aws-lambda/hello-world-python3.png
new file mode 100644
index 000000000..f8a9a81fa
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/hello-world-python3.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/lambda-code-editor.jpg b/static/img/210406-python-sentry-aws-lambda/lambda-code-editor.jpg
new file mode 100644
index 000000000..4d46f93c7
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/lambda-code-editor.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/lambda-search-bar.png b/static/img/210406-python-sentry-aws-lambda/lambda-search-bar.png
new file mode 100644
index 000000000..8388871f3
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/lambda-search-bar.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/layer-with-arn.png b/static/img/210406-python-sentry-aws-lambda/layer-with-arn.png
new file mode 100644
index 000000000..70033365b
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/layer-with-arn.png differ
diff --git a/static/img/210406-python-sentry-aws-lambda/sentry-dsn-string.jpg b/static/img/210406-python-sentry-aws-lambda/sentry-dsn-string.jpg
new file mode 100644
index 000000000..69a340fb8
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/sentry-dsn-string.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/sentry-error-dashboard.jpg b/static/img/210406-python-sentry-aws-lambda/sentry-error-dashboard.jpg
new file mode 100644
index 000000000..4fd7def93
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/sentry-error-dashboard.jpg differ
diff --git a/static/img/210406-python-sentry-aws-lambda/sentry-homepage.jpg b/static/img/210406-python-sentry-aws-lambda/sentry-homepage.jpg
new file mode 100644
index 000000000..0fe92307a
Binary files /dev/null and b/static/img/210406-python-sentry-aws-lambda/sentry-homepage.jpg differ
diff --git a/static/img/210823-sentry-apm-lambda/arn-region.png b/static/img/210823-sentry-apm-lambda/arn-region.png
new file mode 100644
index 000000000..4bca3e021
Binary files /dev/null and b/static/img/210823-sentry-apm-lambda/arn-region.png differ
diff --git a/static/img/210823-sentry-apm-lambda/layer-with-arn.png b/static/img/210823-sentry-apm-lambda/layer-with-arn.png
new file mode 100644
index 000000000..577cbba74
Binary files /dev/null and b/static/img/210823-sentry-apm-lambda/layer-with-arn.png differ
diff --git a/static/img/210823-sentry-apm-lambda/performance-results.jpg b/static/img/210823-sentry-apm-lambda/performance-results.jpg
new file mode 100644
index 000000000..0e9d3f96d
Binary files /dev/null and b/static/img/210823-sentry-apm-lambda/performance-results.jpg differ
diff --git a/static/img/210823-sentry-apm-lambda/performance.jpg b/static/img/210823-sentry-apm-lambda/performance.jpg
new file mode 100644
index 000000000..7871121d7
Binary files /dev/null and b/static/img/210823-sentry-apm-lambda/performance.jpg differ
diff --git a/static/img/210823-sentry-apm-lambda/sentry-dsn-string.png b/static/img/210823-sentry-apm-lambda/sentry-dsn-string.png
new file mode 100644
index 000000000..873e68480
Binary files /dev/null and b/static/img/210823-sentry-apm-lambda/sentry-dsn-string.png differ
diff --git a/static/img/fsp-fav.png b/static/img/fsp-fav.png
index 28a9f6cdb..3a2f75910 100644
Binary files a/static/img/fsp-fav.png and b/static/img/fsp-fav.png differ
diff --git a/static/img/headers/python-lambda-sentry.jpg b/static/img/headers/python-lambda-sentry.jpg
new file mode 100644
index 000000000..18f0d40db
Binary files /dev/null and b/static/img/headers/python-lambda-sentry.jpg differ
diff --git a/static/img/logos/cloudflare.png b/static/img/logos/cloudflare.png
new file mode 100644
index 000000000..b6a18d47e
Binary files /dev/null and b/static/img/logos/cloudflare.png differ
diff --git a/static/img/logos/oracle.jpg b/static/img/logos/oracle.jpg
new file mode 100644
index 000000000..6e8fec361
Binary files /dev/null and b/static/img/logos/oracle.jpg differ
diff --git a/static/img/visuals/cx-oracle.jpg b/static/img/visuals/cx-oracle.jpg
new file mode 100644
index 000000000..878e7d285
Binary files /dev/null and b/static/img/visuals/cx-oracle.jpg differ
diff --git a/static/img/visuals/oracle-orm-examples.jpg b/static/img/visuals/oracle-orm-examples.jpg
new file mode 100644
index 000000000..adae67764
Binary files /dev/null and b/static/img/visuals/oracle-orm-examples.jpg differ
diff --git a/theme/templates/base.html b/theme/templates/base.html
index f477ca1c7..93e3ad132 100644
--- a/theme/templates/base.html
+++ b/theme/templates/base.html
@@ -17,9 +17,8 @@
{% block content %}{% endblock %}