Here are the slides from a talk I gave today at the BibleTech conference in Seattle. Download the accompanying handout or explore the Expanded Bible interface mentioned in the presentation.
Archive for the ‘Topics’ Category
Designing for Agency in Bible Study
Saturday, April 13th, 2019Topics for the U.S. presidential election
Friday, November 18th, 2016The below chart lists the most-popular topics on the topical Bible on the days after the U.S. presidential elections in 2012 and 2016.
* “End times” includes the following topics: signs of the end times, end of the world, end of days, the end of times, in the last days, last days, the apocalypse, judgment day, america in the end times. “Honoring leaders” includes: honoring leaders, submission to human authority, obeying authority, respecting authorities, respecting those in leadership. “World government” includes: world government, new world order.
Topping the list this year is “end times,” which includes several related topics. The “end times”-related “the antichrist” and the 1990s-vintage “world government” also jumped a number of places. The Google Trends report for end times shows spikes around the 2012 and 2016 elections, though world events (especially in the Middle East) regularly increase “end times”-related searches.
“Power of words” also jumped up a few places; during the October 7 news cycle around the Donald Trump/Billy Bush video, “power of words” was the fourth-most-popular topic, behind voting, marriage, and forgiveness.
Christianity Today puts this data in more context in their article The No. 1 Bible Verse and Top 25 Topics of Trump’s Election.
How the Internet responded to the Supreme Court same-sex marriage decision on Google, Twitter, and Bible Gateway
Monday, June 29th, 2015Evangelical leaders: “Outrage and panic off-limits.” Internet responds with outrage and panic.
I was half-hoping that Christianity Today would use this Buzzfeedy title (referring to a previous article of theirs) for my piece there about the quantitative aftermath of Friday’s Supreme Court decision.
Here’s a teaser:
Separately, the same-sex-marriage topics I mentioned in my last post have received a surge of votes since the decision on Friday, and I’m pleased (from a statistical perspective) that the voting is consistent with previous patterns.
The below chart shows the cluster voting pattern since 10 AM EDT on June 26, 2015, when the decision was announced. The three clusters with the lowest number of overall votes didn’t receive any votes during this period: “Everything for its purpose,” “David and Jonathan,” and “Personal struggle against sin.”
Quantifying how people cite the Bible in the same-sex marriage debate
Friday, June 26th, 2015A recent New York Times feature discusses how evangelicals interpret a few Bible passages in support of or against same-sex marriage. The pro-con format of the Times feature supplies anecdotes, but it doesn’t answer the question of how people are applying the Bible to the debate on a wider scale. For that, we have to turn to data.
This site has a topical Bible that allows people to vote on and suggest Bible verses they think are relevant to a particular topic. Since 2007, the topics of homosexuality, same-sex marriage, and gay marriage have received over 30,000 votes across 940,000 pageviews, providing a reasonably broad window into the passages that people–largely evangelical Christians–apply to the debate.
The statistical patterns of these 30,000 votes reveal sixty Bible passages that commonly enter the discussion, falling into ten thematic clusters:
This chart shows the strongest statistical connections among the ten clusters.
Three of these clusters are used against same-sex marriage: nineteen passages about sexual immorality (the largest cluster), four relating to the Old Testament Law, and three about man and woman becoming one flesh in marriage.
Four clusters are used to justify acceptance (or at least tolerance) of same-sex marriage: seven passages about loving your neighbor, five about David and Jonathan, four about not judging, and four about valuing eunuchs and the childless.
The final three clusters don’t deal directly with the topic: ten passages about living morally, two about how everything has a purpose, and two about Paul’s personal struggle with sin.
We can also examine how the popularity of each cluster has changed since 2011:
(This chart uses a logarithmic scale to better distinguish the lower vote counts; the top lines have much higher vote counts than the lower lines.)
This data is hardly scientific since it depends on visitors to a single website and doesn’t necessarily indicate culture-wide rhetorical shifts. With that caveat in mind, however, we can observe a few trends.
First, the three clusters with the most votes are all against same-sex marriage. The “one flesh” cluster gained speed in March 2013, when the U.S. Supreme Court heard arguments about California’s Proposition 8 and the federal Defense of Marriage Act.
Second, the three most-popular clusters in support of same-sex marriage started earning substantial numbers of new votes at the end of 2013, when Hawaii, Illinois, and New Mexico legalized same-sex marriage, and a judge ruled Utah’s ban unconstitutional. A Christianity Today article from the time indicates that at this point evangelicals started to recognize that same-sex marriage would probably become legal everywhere and began to change their rhetorical strategies. The above chart suggests a shift in tone at this time away from “do not judge” to “loving your neighbor” and applying to the debate passages about accepting eunuchs and the childless.
Third, the chart shows a more-recent (and still nascent) rhetorical shift to comparatively ambiguous positions. The general command to “live morally” can apply to both sides of the debate, for example.
These shifts are consistent with recent polls that suggest opposition to same-sex marriage is softening among evangelicals. Just looking at the topical Bible vote counts above, in October 2013, 95% of votes were for passages used against same-sex marriage; by May 2015, this percentage had fallen to 72%.
These trends suggest to me that many people who use Bible verses in the debate are adapting to a shifting rhetorical landscape. They’re attempting to discern how the Bible can remain relevant to the conversation and are in turn changing the mix of passages they cite.
Venn Diagram of Google Bible Searches
Monday, October 25th, 2010Technomancy.org just released a Google Suggest Venn Diagram Generator, where you enter a phrase and three ways to finish it: for example, “(Bible, New Testament, Old Testament) verses on….” It then creates a Venn diagram showing you how Google autocompletes the phrase and where the suggestions overlap.
The below diagram shows the result for “(Bible, New Testament, Old Testament) verses on….” The overlapping words–faith, hope, love, forgiveness, prayer–present a decent (though incomplete) summary of Christianity.
Topical Bible Technical Notes
Monday, July 2nd, 2007As promised, here’s how the new Topical Bible works.
The Goal
The goal was to create a new topical Bible (TB) that takes advantage of the vast array of data on the Internet, warts and all.
Seeding the Topics
I wanted the TB to reflect what people really want to know, not what a human editor thinks people want to (or should) know. (I have no problem with human editors, but the point of the TB is to see what the results are when you forego them.) That meant populating the TB with actual queries. But where to get them? The options:
- Start with a few topics hand-selected by me based on research and intuition. The problem is that the number of topics would be small (probably less than 100). That number would (hopefully) increase over time, but it would grow erratically and might not attract enough people to the TB to make it useful. Further, the wording of the topics would reflect my biases.
- Start with the topics from a public-domain Nave’s Topical Bible. The main problem is that a lot of the “topics” are obscure, often just names of biblical people. Further, English has changed in the 100 years since Nave published his TB; how many people search for Bible verses about abstemiousness? So I’d either have to clean up the topics or live with a lot of irrelevant topics.
- Ask for permission to license a topic list from the publisher of a newer TB or from another TB website. They might be willing to share their list. But again, the problem is that the topics wouldn’t reflect what people really want to know about.
- Use a search engine API to generate the topics.
In the end, I combined the Yahoo Related Suggestion API and Firefox’s auto-complete feature (completing the phrase “What does the Bible say about…”) to create a list of about 4,000 topics.
Some of the resulting topics had typos, so I ran them all through the Yahoo Spelling Suggestion API, which turned up 176 misspelled topics (and a few misses—no, I really did mean “caring for widows,” not “caring for windows”).
Getting Related Verses
The next step was to get the verses. I used the Yahoo Web Search API to get the top thirty webpages related to each topic and then extracted the verse references from each page.
Daniel Foster from Logos rightly points out that extracting Bible verses from webpages is “fraught with perils.” Thankfully, it doesn’t have to be perfect; it just has to be good enough.
The ESV folks have published the Bible-book abbreviation list they use on their site. I started with the abbreviations in that file, then built up regular expressions to find only fairly definite Bible references. (For example, references to some person named Matt shouldn’t match a Bible reference.)
I stripped the HTML from the webpage and did some more normalizing, then went through each of the abbreviations (the actual code is a little more complex, but you get the idea):
my $ref = '\d{1,3}(?:[.:]\d{1,3})?(?:\s?[\&\-]\s?\d{1,3}(?:[.:]\d{1,3})?)?[ab]?'; #chapter/verse references
my $verse = "$ref(?:\\s?[,;]\\s?(?:$ref))*"; #multiple verse references
foreach my $abbrev (@abbrevs)
{
my $regex = "\\b$abbrev\\.? ?$verse\\b"; #go through each abbreviation
while (/$regex/) {…}
}
Next I figured out exactly which verses each reference refers to. For example, the string “Genesis 1,2” really means “Genesis 1:1-31 and Genesis 2:1-25.” The ESV API’s getQueryInfo method figures out everything for me. Why should I write a bunch of reference-parsing code when someone’s done all the hard work?
So I did an ESV API query for each reference, caching the results so identical references in the future don’t require an API lookup.
Collating the Verses
Once I retrieved all the verses for a topic, it was just a matter of looking for patterns among all the webpages. The algorithm was pretty simple: each page got one vote per unique verse—so two references to John 1:1 on the same page would only count as one vote. All verses that appeared on two or more webpages made it into the main TB index.
Sweetening the Relevance
In the end, I was able to use some of the topics from Nave’s work. About 750 of the topics occurred in both the new TB and in Nave’s; every verse for each topic in Nave’s got an extra three votes in the new TB. So, for example, 1 Corinthians 15:45 originally had eight votes under the topic of Adam. The mention in Nave’s added another three votes, bringing the vote total to eleven votes (to start with).
Displaying the Bible Verses
I use the ESV API to display the Bible verses, with heavy caching. Only five verses appear when the reference is to an extended passage, though the link points to the complete passage on the ESV site.
A recent addition to the UI is a “related topics” link—a reverse TB that lets you see the all the topics for a given verse in tag-cloud format. Entering a verse reference into the search box takes you to the same page. For example, here are all the topics for Galatians 5:13.
Ongoing Topics
The site follows a simplified version of the above procedure for new words: anytime someone searches for a word that doesn’t exist, the site goes out and finds relevant pages, parses them, and displays the results.
I originally wrote a nice multithreaded Perl script to fetch new topics quickly, but Dreamhost (my hosting company) kills it when it runs, so now everything happens serially—thus slowly.
Weaknesses
It works pretty well for popular topics. My favorite user-created topic is Christian Hedonism, a phrase popularized by John Piper; the TB did particularly well for this topic.
It doesn’t work so well for obscure topics or topics about which the Bible doesn’t really have anything directly relevant. More sophisticated algorithms might be able to correct this deficiency to an extent, but I’m not sure what form those algorithms would take.
Daniel from Logos rightly points out that the TB should really be “What people say the Bible says about….” I find some of the verses chosen for topics personally offensive. Do you really want to tell people with eating disorders that the weak person eats only vegetables, and not to judge them if they abstain from eating?
The main danger is in taking verses out of context; I’m afraid that someone will read a verse and act on it without considering what the verse means in context.
Results
Since launching three weeks ago, people have voted up or down 3,000 verses and suggested 200 new verses, in addition to creating 500 new topics.
Why?
The recent book Everything Is Miscellaneous inspired the TB. The book has a number of implications for Bible study; I may blog about it in the future. I wanted to apply some of the book’s ideas to the Bible, and the TB seemed like an easy way to do it.
In all, it took about three weekends of part-time work to create, somewhat less than the fourteen years Orville Nave spent on his famous TB. Collecting the data was the easy part; coding the frontend took several days.
The Future
Daniel from Logos has a few suggestions about letting people tie together topics instead of creating new ones for each query. I can (and do) edit the database to reduce some of the redundancy. I’ve given some thought about the UI and backend for such a system and how to prevent people from manipulating it, but an implementation remains in the future. I’d also like to spell-check new topics and let people correct the spelling instead of automatically creating the topics.
It would be helpful for people to be able to tag verses with topics when looking at a verse’s tag cloud. The main issue is how to flow the topics back into the topical index.
I’m also not entirely satisfied with how you have to know the exact verse reference to suggest a verse for a topic. Ideally, you’d also be able to enter a few keywords and get back relevant verses.
Overall, however, the TB has performed well so far.
New Feature: Topical Bible
Saturday, June 9th, 2007It’s really more of a web 2.0 topical Bible mashup. Some popular topics: marriage, divorce, gambling, dating, Harry Potter. (Try finding Harry Potter in your Nave’s Topical Bible.)
The topical Bible combines the Yahoo! and ESV Bible web services to identify topics and find relevant verses. It currently has about 4,000 topics. Searching for a topic that doesn’t exist will automatically add it to the topical Bible, which will then scour the Internet for relevant verses.
You can vote on whether the listed verses are relevant to the topic—or you can suggest new verses. Over time, the topical Bible should become better at identifying what people think the Bible says about a topic.
More technical details will follow, but I wanted to post it and let you try it out. It’s still a work-in-progress, but what isn’t these days?
Here’s a screenshot:
