PIEGraph FAQ

ABOUT THIS SITE [back to table of contents]

  • What is this site, and what is its purpose?
  • Why are you doing this?
  • Who is in charge of this project?
  • Do I have to be a research participant to use the site?
    • These questions are answered on our “About Us” page

ABOUT THE DATA [back to table of contents]

  • Where did you get the political bias and factual content data from?
    • We obtained the data about political bias and factual content from the independent reporting outlet Media Bias/Fact Check (MBFC). The data were provided free of charge by MBFC for the purposes of this study.
  • What is MBFC, and how credible is it?
    • MBFC is a U.S.-based, independent reporting outlet established in 2015. It provides free information about media bias, including political bias and factual content, and occasional fact checking services to users of its website.
    • MBFC states that it uses both objective measures and subjective analysis to determine political bias and factual content rankings. The site publishes the methodology it uses to determine rankings for review by site users. It follows the International Fact-Checking Network Fact-Checkers’ Code of Principles developed by The Poynter Institute and consistently makes the lists of reliable bias- and fact-checking sources. MBFC earned a perfect score for credibility from Newsguard, and its data has been used in multiple peer-reviewed studies, including Aires et al. (2019), Stefanov et al. (2020), and Cinelli et al. (2020).
  • What do you mean by “factual content”?
    • Factual content includes elements of trustworthiness and expertise. MBFC assigns a factual content rating based on a combination of factual reporting, website traffic and whether a source is considered questionable, conspiracy or pseudoscience. MBFC’s factual content ratings are: Very High, High, Mostly Factual, Mixed, Low, and Very Low. More detailed information about how these categories are determined is available on the MBFC Methodology page.
  • What do you mean by “political bias”?
    • In its most basic definition, ideology is simply a system of ideas that contributes to political beliefs and ways of thinking. In the United States, ideologies often map onto political (e.g., Left/Right) biases. The ideology of liberalism (i.e., the Left) in the broadest sense, for example, is often equated with human rights and equality. Conservatism (i.e., the Right), on the other hand, is often equated with limiting the power of the federal government and respect for American traditions. Another example, from MBFC, is that the Left values collectivism or the importance of community over individual well-being, while the Right values the freedom of the individual over the good of the community.
  • How did you determine the political biases of your sources?
    • Some media outlets express a Left/Right political bias in their reporting. To determine where an outlet falls on the ideological spectrum, MBFC examines the use of biased wording and headlines, the appearance of well-sourced, factual, and balanced reporting, and if an outlet strongly endorses a particular political party or ideological stance. More detailed information about how MBFC determines Left/Right bias is available on its Methodology and Left vs. Right Bias pages.
    • Political bias is also sometimes referred to as “political affiliation.” Political affiliation is a term MBFC uses to describe the extent to which a source endorses a political ideology like liberalism, conservatism, collectivism, or individual rights. MSNBC, for example, tends to favor a more liberal ideology and, therefore, has a more Left-leaning political affiliation, Fox News, on the other hand, tends to favor a more conservative ideology and, therefore, has a more Right-leaning political affiliation.
    • There are media outlets (e.g., Reuters) that MBFC deems “least biased” in terms of political affiliation. This does not mean they are neutral but that they mostly present ideologically balanced news reports.
  • What do you mean by “Left,” “Center Left,” “Center Right,” “Right,” etc.?
    • These labels come from MBFC, which defines them as:
      1. Least Biased: The most credible sources. Minimal bias, very few loaded words and emotional appeals. Factual, usually well-sourced, reporting.
      2. Left Bias: Moderately or strongly biased to liberal causes. Use loaded words and emotional appeals. May publish misleading reports, omit information, or use untrustworthy sources in reporting.
      3. Left-Center Bias: Slightly or moderately biased to liberal causes. Often publish factual information but use loaded words and emotional appeals. Generally considered trustworthy overall but may require additional investigation on your part.
      4. Right-Center Bias: Slightly or moderately biased to conservative causes. Often publish factual information but use loaded words and emotional appeals. Generally considered trustworthy overall but may require additional investigation on your part.
      5. Right Bias: Moderately or strongly biased to conservative causes. Use loaded words and emotional appeals. May publish misleading reports, omit information, or use untrustworthy sources in reporting.
  • Why are so many more of the Right-wing sites rated as “low factual content” compared to Left-wing sites? Aren’t you just using the idea of “factual content” to advance your own Left-wing politics?
    • No. MBFC publishes details about its methods for determining credibility scores on its Methodology page. As it points out, there is no scientific formula that is 100% objective to determine a media outlet’s credibility score. However, MBFC is transparent in its methodology and uses a combination of objective measures and subjective analysis in its rankings. Additionally, MBFC has a mechanism for correcting factual errors. Ultimately, individual acceptance of MBFC’s factual content scores requires
      1. General agreement with traditional academic and journalistic standards of empirical evidence-gathering, and
      2. Rejecting conspiratorial thinking and information seeking strategies driven by confirmation bias.

ABOUT OUR RESEARCH [back to table of contents]

  • If I was recruited by Dynata or one of its affiliates to participate in the research study, how will my data be used? Will I be notified when any publications based on my data come out?
    • No personally identifiable information will be collected if you participate in this research study, and the research team will not have access to any such information Dynata may have about you. The only information the research team will store is your anonymous survey answers and the link domains from your Twitter feed containing media outlet sources.
    • We are interested in understanding the extent to which, collectively, people are exposed to sources of information that vary in political bias and factual content. The results of this study will help society better understand how ideology and information quality intersect to determine the content people see on Twitter.
    • If you choose to participate in the study, you will be able to view your own personal information environment. This “PIEGraph” is a visual representation of ideological and information credibility that is in your Twitter feed based on the hyperlinks from media sources that appear there.
    • It is likely the results of the study will be presented at academic conferences and/or to the public, as well as published in peer-reviewed journals and possibly in the popular press. Participants will not be notified when such presentations or publications occur. However, no personally identifiable information will be disclosed about you.
  • If I am not enrolled in the study and I just signed up on my own, how will you use my data?
    • Unless you signed up through Dynata, we will not use your data in any published study. However, we may use your data for unpublished preliminary research.
  • What information does PIEGraph collect about me and/or my Twitter timeline?
    • PIEGraph does not collect anyone’s personally identifiable information. The system tracks users through an anonymous user ID that cannot be traced back to you or to your Twitter account. If you were recruited by Dynata to participate in this study, Dynata assigned you this ID, which is passed through each step. In other words, PIEGraph does not collect your Twitter username, password, profile image or description.
    • The only information PIEGraph is collecting as you use Twitter is the links that appear in your timeline that originated with one of the media outlets that appears on MBFC. This could be a link Tweeted directly from one of these media outlets, or it may be a link Tweeted by someone you follow containing a link from a media outlet.
    • Your individualized collection of Tweeted links is associated with your anonymous user ID.
  • May I view this study’s informed consent form?
  • What else do I need to do for the study after I grant your app access to my Twitter timeline?
    • Nothing, you are done. Dynata should send you your compensation within the next two weeks. If they do not, please contact them–we cannot help you as we have no information on our user’s identities.

USING PIEGRAPH [back to table of contents]

  • How many different sites will show up on this chart? Is there a complete list somewhere?
    • The sites that appear in your personal information environment graph (PIEGraph) were sourced from the MBFC database. The MBFC database includes more than 3,000 media sources and journalists. If a source or journalist is not in the MBFC database, it will not appear on your PIEGraph. All included sites are listed on the MBFC website.
  • What do the X- and Y-axes mean?
    • The X-axis on PIEGraph is the horizontal line labeled “political bias.” Its midpoint is denoted by the number 0. Media outlets falling to the left of 0 on the Y-axis have a Left-learning or more liberal political orientation, whereas outlets falling to the Right of the 0 have a more Right-leaning or conservative political orientation. The farther away from 0 a media outlet lies, the more politically-biased MBFC judged it to be. The Y-axis is the vertical line labeled “factual content.” Its midpoint is denoted by the number 0 (i.e., the middle horizontal line). Media outlets appearing above 0 on the X-axis contain more factual content, with 0 corresponding to an MBFC rating of “Mixed” factual content. Slightly more than half of all sites are rated between 1 and 3 (“Mostly Factual,” “High,” or “Very High,” respectively), while about 41% of sites have a rating of 2.
    • In general, the Y-axis (factual reporting) is more important than the X-axis (political bias). The Pittsburgh Post-Gazette, for example, has a slight Right political bias but contains factual reporting. Similarly, The Atlanta Journal-Constitution has a slight Left political bias but also contains factual reporting. We believe that an outlet’s factual reporting score is more important than its ideological stance.
  • What does it mean if a site falls below the X-axis? Does it mean they lie more than they tell the truth?
    • MBFC describes sites with a factual content score of 0 (i.e. exactly on the X-axis) as “not always using proper sourcing or sources to other biased/mixed factual sources. Mixed sources will have failed one or many fact checks and do not correct false or misleading information.” Sites with scores of -1 or -2 “rarely use credible sources and [are] not trustworthy for reliable information.” The site describes its full methodology for determining factual content scores here.
  • What do the numbers on the chart mean (e.g., -2 to 3 for factual content; -33 to 33 for political bias)?
    • We describe how we generated these numbers and what they mean on our “Methods and Data” page.
  • What do the sizes of the nodes (bubbles) mean in PIEGraph?
    • The bigger the size of a media outlet’s node, the more often that outlet appears in your Twitter timeline. You can see the exact number of times an outlet has appeared using the “Chart Data” link at the bottom left of the main page. The number in the line that reads “Number of Occurrences” is the number of times that outlet has appeared in your Twitter timeline.
  • Why do some sources not appear in my personal information environment? Can they be added?
    • The two most common reasons why a source may not appear in your personal information environment are: (1) that source appeared outside of the time window from which PIEGraph is collecting sources, and (2) the source does not appear in the MBFC database.
    • Missing sources will not be added unless MBFC adds them to its database and assigns scores to them. Adding and scoring them ourselves would compromise the validity of the study.
  • Why doesn’t PIEGraph display content from all my Tweets instead of just ones with media links in them?
    • We are interested in the political biases and factual content of the linked media sources you see in your Twitter feed. We hope this will help us better understand the extent to which people are exposed to information that varies across these dimensions. We focus on media sources because the media shapes public opinion of topics.
  • How do I read the mean, median and standard deviation scores under Tweet Statistics?
    • The mean is the average of all the scores for a given category in your Tweet statistics. The mean is calculated by adding all the scores and dividing by the number of scores. For example, if you have 3 scores (1, 2 and 3), you would add them all up and divide by 3. For example (1+2+3) / 3 = 2
    • The median is the middle point in your list of scores. For example, in this list: 3, 5 7, 9, 11; 7 is the median value.
    • The standard deviation describes how scores are dispersed across a distribution. In lay terms, it tells you how far your score is from the mean (or average) of all other scores in the study. Wikipedia includes a good explanation of what this term means statistically.
  • Who else other than me can see my personalized PIEGraph chart?
    • Only someone with your Twitter login information can see your personalized PIEGraph chart. However, the IRB-approved PCAD research team has access to the data displayed in your PIEGraph chart. This data does not include any personally identifiable information about you. For more information about personally identifiable information, see our answer to “What information does PIEGraph collect about me?”
  • What can I do with the information contained in my personal information environment?
    • We hope PIEGraph will help promote healthy information consumption practices by providing a visual representation of the political bias and factual content scores of media sources that appear in your Twitter feed.
    • In general, higher rates of factual content matter more than the political bias of a media source. Therefore, if you see many media sources that appear below the X-axis in your PIEGraph, you may need to be concerned about misleading content appearing in your Twitter feed and further investigate claims with sources containing more credible content.
    • Similarly, you may want to adjust your media diet and actively seek out more information from those media sources that appear on or above the X-axis.
    • It may also be a useful practice to look at stories about the same topic across multiple media sources that fall above the X-axis but to the left or the right side of the Y-axis for political bias. For example, you may want to read a story about climate science in the Washington Post, which rates a 1 for factual content and a -15 for political bias (Center Left), and then look at a story on the same topic in Forbes, which also rates a 1 for factual content and a -14 for political bias (Center Right).
    • You can also compare your credibility and bias scores with those of other PIEGraph users by clicking on the Tweet Statistics tab. This tab contains the mean, median, and standard deviation for your personal information environment (under My Credibility and My Bias) and for the PIEGraph community (under PIE Credibility and PIE Bias). These statistics allow you to compare your own information environment with others enrolled in the study. Are the media sources shared in your Twitter timeline more or less factual or more right- or left-leaning? If you don’t want to bother with the statistics, we have also included a brief statement summarizing this for you (e.g. “36% of users have more left-wing content than you, and 64% have more right-wing content. 36% of users have a higher factual content average than you, and 64% have a lower factual content average”).
  • My personal information environment contains many sources with low factual content and/or high Left/Right Bias scores. What can I do about that?
    • Below your personal information environment graph (PIEGraph), there is a table with the headings: Chart Data; Home Timeline Tweets; and, Tweet Statistics. Each of these headings can be filtered by date, political bias, factual content, source and more.
    • If you want to remove sources that contain low factual content or high Left/Right bias, we recommend navigating to the Home Timeline Tweets heading and sorting the results you see by political bias and/or factual content ranking.
    • Decide what threshold you believe is too low in factual content or too far Left or Right in your opinion. Perhaps, for example, you view anything below 1 on the Factual Content axis as too low. In this case, you can scroll to the right to view all the Screen Names that have shared Factual Content rated below 1. You may choose to unfollow these screen names. However, a few words of warning: (1) the Twitter user who posted this content may be critiquing this media source because they feel it is not credible (2) this one Tweet may not be representative of the user’s whole feed; for this reason, you may prefer to unfollow only those users who repeatedly post content below the threshold you have set for yourself (3) unfollowing a Twitter user does not guarantee a given media source will not show up in your PIEGraph again since someone else may share a link from it.

NEED SUPPORT OR HAVE QUESTIONS FOR US? [back to table of contents]

  • I’m having a technical issue with PIEGraph or have a general question, comment, or suggestion. How can I get an answer?
    • Please email pcad.help(at)gmail.com with your question or a description of your problem and we will address it ASAP.
  • How can I disconnect PIEGraph from my Twitter feed?
    • To disconnect PIEGraph from your Twitter feed:
      • Login to your Twitter account
      • Navigate to the “Settings and Privacy” section of your account
      • Click on “Security and Account Access”
      • Click on “Apps and Sessions”
      • Click on “PCAD PIE”
      • Click “Revoke App Permissions.” You are now disconnected, and PIEGraph will collect no further information from your Twitter feed.
  • I disagree with how a particular site was scaled.
    • Okay, but that’s not a question.
  • Will you change how that site is scaled based on my disagreement?