Knowledge of the Book of Mormon

How much do Mormons really know about the Book of Mormon? Have we immersed ourselves in the text to the point where quotations from it are immediately recognizable, like long-lost friends? Or is our experience of the text more like attending our spouse’s family reunion, where we have a handful of very close connections but a much larger number of people who may be vaguely familiar but to whom we still need to be introduced?

As most readers know, I recently conducted a large internet-based survey of the Book of Mormon experiences of more than 2000 people in the broader Mormon world. The results from this survey need to be thought about with care. The sample in question is not random. Furthermore, it is clear from some of the results in the data that it is not representative of the Mormon community more generally; people in this sample are better educated than the Mormon world in general, and they also spend more time thinking and reading about Mormonism than is the norm. In other words, we ought to expect the people in the sample to have greater textual familiarity with the Book of Mormon, on average, than would people in the broader Mormon world. However, definitive conclusions about that broader world await the completion of a survey with a much better (and more expensive) sampling design; the current results may be of interest both because they provide some data about questions that are otherwise hard to resolve, and because the sample captures the experiences of a group of mostly very engaged members of the broad Mormon community — the kind of people who are particularly likely to serve as opinion leaders.

What can the survey tell us about people’s knowledge of the Book of Mormon? The survey contains nine items intended to measure people’s familiarity with the text of the Book of Mormon and with related text-critical discourse. Three of these ask the reader to identify the speaker in quotations from the book, while another four ask the reader to determine whether a specific quotation is from the Book of Mormon or another scriptural text. One question asks people to identify “chiasmus” from a definition, while the final item asks people to specify the proportion of the Book of Mormon text that consists of quotation of KJV Bible language. These items were not all equally successful; more on that in a moment.

First, let’s look at simple description of respondents’ knowledge of the Book of Mormon text. The following graph shows the overall percentage of respondents who were able to correctly identify the speakers in each of three quotations (one from Jacob talking about the importance of Isaiah; one from King Benjamin describing various details of Jesus’s life; and one from Moroni promising readers that they can know the truth of God’s message through pondering and prayer).

July_2008_speakerident1

A first set of remarks relates to the definition of a correct answer. Spelling doesn’t count in this evaluation; so, for example, “Maroni,” “Mroni,” “Moronim,” and “Moroin” were all correct answers for the third question. The one individual who answered the second question with the strangely colloquial “King Benji” was also interpreted as having given the correct answer. Likewise, editorial comments didn’t interfere with an otherwise correct identification. Responses such as “Joseph Smith Writing as Jacob,” “Allegedly King Benjamin but really it’s Satan,” and the ever-charming “Horny Joe as Moroni” are all counted as correct. I am unsure which of these two rules to cite when discussing the one respondent who labeled the third quote as being a product of “Moron,” but the answer was acceptable in either case.

Now a comment on the levels. The Moroni quote is clearly well-known in Mormon circles; all current and former missionaries as well as all investigators probably know the quote by heart. So also anyone who went through the church’s youth programs. This ubiquity is reflected in the 78% correct response rate on this passage in the survey. Most people who misidentified this passage labeled it as a text written by Mormon, while some simply stated that they couldn’t recall. The other two passages are clearly much less well-known; only 15% correctly identified Jacob as the speaker in the first instance, and about 20% were able to identify Benjamin as the speaker in the second case. For the Jacob case, many people confused this with Nephi’s comments on Isaiah later in 2 Nephi — a highly understandable mistake. The Benjamin passage was less readily recognizable; most incorrect responses simply provided some variant of “I don’t know.”

Another set of questions asked respondents to identify the book of scripture from which four passages were drawn. The first quotation is drawn from the KJV translation of the letter to the Hebrews, the second from 3 Nephi 27; the third from Abinadi’s discourse on the Godhead in Mosiah 15; and the fourth from Stephen’s speech at his martyrdom from Acts 7. In some ways, this was an easier task than identifying specific speakers; nonetheless, it was challenging enough, as seen in the following table showing respondents’ success rates at identifying the various passages.

July_2008_rightbook

These results suggest that people in the Mormon world may be more familiar with the Book of Mormon than with the New Testament; a plausible alternative interpretation might be that the Book of Mormon has a more immediately identifiable verbal feel than the KJV New Testament. In either case, substantial numbers of respondents misidentified each of the passages under consideration.

For the two remaining textual knowledge questions, the results are easily summarized. About 58% of respondents correctly identified chiasmus from its definition — suggesting that this task is slightly easier than identifying the 3 Nephi passage as being from the Book of Mormon, although somewhat harder than identifying the Mosiah passage as a Book of Mormon text or recognizing Moroni’s promise. Clearly, this particular textual idea has become substantially widespread in our community. For the question about quotes from the KJV, results were as shown in the figure below:

July_2008_kjvquotes
On average, respondents feel that there is a small to moderate amount of quoting from the KJV in the Book of Mormon; few respondents would claim that there are no quotations, while a relatively small proportion see more than a quarter of the book as quotation from the Bible. In the end, this indicator performed poorly, showing weak relationships with the other textual knowledge indicators no matter how it was coded. This is likely due to ambiguity regarding the definition of a “quote” for present contexts. Does acknowledging quotation of the KJV in the Book of Mormon imply plagiarism? Do quotes count if certain minor words or phrases within the passage are modified? Does the question refer only to explicit quotations, such as of Isaiah or Amos, or does it also include more implicit quotations such as the language from Hebrews that some scholars have found in Alma 13? My question text resolved none of these issues, which is likely the reason that this indicator failed.

A central purpose of this survey was to explore the measurement characteristics of survey items about the Book of Mormon, in order to help develop useful and empirically validated items for researchers interested in Book of Mormon experiences. A useful summary tool for these purposes is item response theory, about which a relatively nontechnical introduction can be read here. Roughly speaking, IRT provides statistical models that estimate the “difficulty” of an item along an underlying dimension, as well as the trait level on that dimension for each respondent. I don’t think the models should necessarily be taken too seriously, but the resulting graphs provide a quick visual summary of patterns that are evident in the data using a variety of techniques. The following graph shows the amount of information about overall textual knowledge found in each of the nine indicators discussed above.

July_2008_textknowledgeiic

Basically, the greater the area under the curve, the more information the survey item provides about people’s textual knowledge of the Book of Mormon. The further the area is to the left, the more information the item provides about high levels of knowledge; the further to the right, the more information about lower levels. Overall, aside from the KJV Quotations item (which corresponds to the basically flat line along the bottom of the graph), everything provides helpful information. If a smaller set of items were needed, I’d recommend using the Hebrews, Mosiah, Jacob, and Acts quotations, possibly as well as the chiasmus item.

IRT also allows us to generate overall scores for each respondent. This allows us to quickly and simply look at relationships between other variables and knowledge of the text. The dark line in the center of each box represents the median level of text knowledge for each category. The boxes represent the interquartile range, while the whiskers extend out to the farthest respondent who is not more than 1.5 times the IQR from the end of the box. Or, in more general terms, the box shows where the middle half of the respondents are, while the whiskers above and below show where pretty much the rest of them fall.

July_2008_knowattend1

Clearly, active church attendance is associated with higher levels of textual knowledge than is the norm for those who attend church about once a month or who are not members. Those who are members but rarely attend fall somewhere in the middle. Perhaps church attendance produces textual familiarity through Sunday School lessons and so forth; perhaps knowing the Book of Mormon text motivates people to be active in the church. At least equally plausible is the hypothesis that the kind of people who read the Book of Mormon carefully and regularly are also the kind of people who are very serious about other church obligations.

Higher levels of education are also substantially associated with higher levels of textual knowledge; without going into further details, educational attainment has a correlation of 0.2032 with knowledge of the Book of Mormon text. This is unsurprising; those who have spent more time being coerced to read texts carefully probably carry the habit over into other domains. Readers of the journal Dialogue know the Book of Mormon better, on average, than nonreaders: the correlation is 0.2090. Regular readership of the FARMS Review of Books is even more strongly correlated with textual knowledge, at a level of 0.3116. Readership of the Ensign is not as strongly related with knowledge of the Book of Mormon text as readership of either of the two more specialized journals, with a correlation of 0.1799. Regular readers of Mormon blogs and message boards, on average, have more knowledge of the text than less frequent readers or non-readers; the correlation falls between that for the Ensign and that for Dialogue, at 0.1984.

Overall, the person most likely to know the Book of Mormon text well would be an active member with an advanced degree who reads the blogs daily and also devours every issue of the Ensign, Dialogue, and the FARMS Review of Books. Forward, Blake Ostler and Kevin Barney! By contrast, the person least likely to know the text well would be a member of the LDS church who never attends, who has not finished high school, and who never reads any of the Mormon periodicals.

Comments

  1. I found it interesting that on the book of scripture query (your second graph above) more people got the Mosiah 15 passage from Abinadi’s discourse on the unity of God correct than any of the other passages. I would have thought this one could have confused many Mormons who would see it as too Trinitarian. Likewise, I was surprised the the Epiphany of Stephen was chosen correctly by the fewest respondents. Perhaps we need to know our New Testament a little better.

    Thanks for providing this — it’s fascinating feedback.

  2. This is great stuff, JNS. I can’t wait for the rest. That, and the concluding paragraph was purely delightful.

    I remember a few questions on reading regularity. Did that have any correlation to textual knowledge?

  3. Very cool JNS. This is a fun set of data you have.

  4. John, I think there are contextual cues in the Mosiah passage that mark it as Book of Mormon speech for people who are even moderately familiar with the text. Where else do you find passages that talk about the Son of God in the future tense? I’m surprised that more people didn’t recognize the Benjamin quote, which I expected to be only moderately challenging.

    J., there’s a positive correlation between frequency of reading the Book of Mormon and knowledge of the text. The correlation is at 0.2446. If we were to adopt naive causal interpretations of these bivariate relations, we’d be forced to conclude that you can learn more about the Book of Mormon text by reading FARMS publications than by reading the text itself. I would not recommend that interpretation.

  5. Cynthia L. says:

    JNS, what is the right answer for the % of quotes from the KJV? I realize it depends on definition, but ballpark?

  6. StillConfused says:

    I need to go to church less so I will score higher

  7. Kevin Barney says:

    I had two questions. First, when people had to identify the “book of scripture” from which a quotation came, was that specific (i.e., “Hebrews”) or generic (i.e., “New Testament”)

    Second, I had the same question as Cynthia L. in no. 5.

    Ironically, perhaps, although I meet all the criteria to be deeply familiar with the BoM text (as you suggest in your final paragraph), I really don’t think that I am. This is probably because I do most of my study topically; I’m not one of those people who have read the text over and over and over from front to back.

    At BYU I had a BoM class, and I always performed poorly on the quizzes that were basically BoM trivia contests. But when the professor assigned a paper, I knocked that out of the park and he just gave me an A for the class based on that, even though my quiz average was terrible.

  8. Mark IV says:

    I have in my possession a copy of the mission statement for the department formerly known as CES. The sattement make it quite explicit that the department’s goal is to promote the memorization of scripture passages through the scripture mastery program. After four years of seminary, a student will have memorized 100 passages. So it is really not surprising that we don’t know the text itself very well.

  9. Cynthia and Kevin, I have to offer three possible correct answers regarding the percent quotations from the KJV in the Book of Mormon text.

    1) If “quotation” is taken as implying “plagiarism” rather than divine origin for the book, then there is no factually correct answer because the question is ideological rather than empirical.

    2) If “quotation” is taken as referring to the explicitly cited introduction of blocks of Bible text, possibly with minor changes, then the correct answer should be in the 5% to 15% range, although the precise figure varies a bit depending on the counting method.

    3) If “quotation” is taken as including #2 but also including the range of non-cited very close parallels between segments of the Book of Mormon and portions of the KJV Bible, the answer should be greater than 25%. How much greater depends on what one is willing to accept as evidence of parallels. If we go as far as to posit that distinctive phrases of only four or five words might count, then the answer can be as high as 40%.

    Kevin, on the book of scripture questions, the response categories were generic; people could choose Old Testament, New Testament, Book of Mormon, etc.

  10. My wife and I delight in reading the Book of Mormon together. We’re currently in the section of missionary efforts of Ammon and his brethren. How I wish that we could have the experiences of King Lamoni and his wife and the others. What power must have attended Ammon’s teaching that the king was willing to, on the spot, give up his former life and hopes that he might obtain what the “son of a liar” promised him.

    Steve

  11. Cool stuff, JNS.

  12. makakona says:

    you might be surprised… even scriptural idiots like me took the survey! i missed any linkage to it here and received the survey link in an email from a far better (religiously) educated person than myself.

    anyway, it was interesting to take and even more fun to see some of the results!

  13. JNS,

    I was surprised when I took the quiz that you picked a scripture from Mosiah 3 because the text you quoted is spoken by an angel to King Benjamin. Of course, King Benjamin is relaying what the angel said, so it is KB speaking in that sense. I wasn’t sure if that ambiguity was intentional or not, but it appears it was not.

    The most surprizing thing to me is that anyone missed the author of “Moroni’s promise.” C’mon people. My favorite answer is “Allegedly King Benjamin but really it’s Satan.” lol

  14. Jacob J., the angel/Benjamin issue actually wasn’t a problem. The speaker in the text is Benjamin, relating words from an earlier vision. Some respondents noted this; only one or two said it was an angel but didn’t specify that Benjamin related the words. For those couple of respondents, I gave them credit anyway. But most people who knew it was an angel also knew it was from Benjamin’s sermon.

  15. In a recent stake conference where Elder Oaks was presiding, in his remarks he said that his memory wasn’t good and he couldn’t quote from memory where many scriptures are located. But with enough key words and with the help of the topical guide, he could find them. That brought a lot of comfort for my otherwise diffuse memory. Interesting exercise, nevertheless.

  16. Left Field says:

    Did you have answers that suggested some people were conflating quote with plagiarize? Those terms seem antonymous to me. Quote implies that you give a citation, which rules out plagiarism. Nearly all of the biblical quotations in the Book of Mormon do cite the original author (e.g., “And now I write some of the words of Isaiah” 2 Ne 11:8). A case can be made for plagiarism of the work of the King James translators and a few short passages of Paul, but all of the major biblical writings (Isaiah, Malachi, and Jesus) are correctly attributed to the same authors those works are attributed to in the Bible.

  17. Left Field, the quote = plagiarism information came from emails, not the survey itself (this, unlike many of the other questions discussed in this post, was a closed-choice question). But emails made clear that people from various ideological positions saw the question as relating to “plagiarism,” by which in this context they seem to have had in mind the claim that Joseph Smith simply copied from the KJV and didn’t have any revelation or inspiration. The answers were clearly in part contaminated by this and in part by the other two possible meanings of “quotation.”

  18. Matt W. says:

    JNS, the problem with going down to 4 or 5 words is that you begin to pick up phrases which contextually are not plagiaristic in nature, but just common phrases. I did an analysis with 8 words and the numbers get much closer to the 5 to 15% range. In fact, the vast majority of 8 words+ quotes are linked in the footnotes.

  19. Matt W., I don’t think the “plagiarism” idea is relevant. Some short phrases are clearly borrowings, as well. Suppose I say, “To do or not to do?” Six short words, all very common, and yet I’m clearly borrowing. In any case, I find your numbers a bit problematic, in that the explicit, cited block quotes themselves are over 10% of the text. Nonetheless, this is a messy area and I agree that there can be a lot of different results.

  20. Hey JNS,

    This is pretty good stuff. I would tend to think that you would really have to be an avid reader of the BOM to pick up on all the BOM quotes except for the Moroni quote.

    Your last paragraph pretty much sums it up.

  21. You may have answered this, but did you consider that it was easy to google the right answers for the citation identifications?

  22. TT, I agree that it is easy to Google the right answers. The text of the survey question asked respondents not to do so, but there’s an honor-system component of this, obviously. In any case, the results suggest that large numbers, at least, didn’t cheat. If there was substantial cheating, the implication would be that the results discussed above are upper limits on people’s textual knowledge of the Book of Mormon.

Follow

Get every new post delivered to your Inbox.

Join 8,416 other followers