Page 3 of 3
1 2 3

Results for sgeapin@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which BEST describes experiment “Interaction Effects” ?Answer Provided: ? When one experiment impacts the outcome of another Correct Answer: When one experiment impacts the outcome of another Comments Entered: NA 2. Which of the following is not one of Cialdini’s 6Continue reading “Results for sgeapin@gmail.com”

Results for sgeapin@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. What is a Sample Ratio Mismatch?Answer Provided: ? When the ratio of alpha and beta aren't as expected Correct Answer: The observed traffic split is statistically significantly different from what was expected Comments Entered: NA 2. Which is NOT a characteristicContinue reading “Results for sgeapin@gmail.com”

Results for deepak.grtiet@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. What should you do before you run an experiment?Answer Provided: ? All answers are correct Correct Answer: All answers are correct Comments Entered: NA 2. “AIDA” stands for which of the following?Answer Provided: ? Attention, Interest, Desire, Action Correct Answer: Attention,Continue reading “Results for deepak.grtiet@gmail.com”

Results for deepak.grtiet@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which BEST defines “Baseline Conversion”?Answer Provided: ? The most recently recorded conversion rate of the metric you want to measure Correct Answer: The most recently recorded conversion rate of the metric you want to measure Comments Entered: NA 2. Which ofContinue reading “Results for deepak.grtiet@gmail.com”

Results for deepak.grtiet@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which is NOT a benefit of having “North Star Metric”?Answer Provided: ? They have no sample size requirements Correct Answer: They have no sample size requirements Comments Entered: NA 2. Which generally happens when testing many variants running at once?Answer Provided:Continue reading “Results for deepak.grtiet@gmail.com”

Event Wrap-up: CONVERT – Conversion Optimisation Meetup (Bristol & Bath)

July saw the first event of CONVERT – a Conversion Optimisation Meetup in the South West of England. 

Attended by 20 people, the evening was packed with interesting discussions around user behaviour tools and success stories.  As with most events, a lot of valuable discussions came from networking and this was the perfect evening for that. Whether it was reconnecting with ex-colleagues or meeting someone with a similar role, it was brilliant to see. I for one had gotten too comfortable behind the screen!

Where should I start with testing?

Like with most processes, starting is usually the most challenging part. You know the feeling. It’s when you don’t have your flywheel going quite yet and you’re not sure at what stage to jump in. When it comes to Experimentation, the usual places I see people jump in are at the Research, Hypotheses creation, orContinue reading “Where should I start with testing?”

Results for jur.schouten@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. What is not a valid CRO option when a site has little traffic?Answer Provided: ? Believe you cannot run any tests Correct Answer: Believe you cannot run any tests Comments Entered: NA 2. Which BEST describes experiment “Interaction Effects” ?Answer Provided:Continue reading “Results for jur.schouten@gmail.com”

Results for jur.schouten@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. When writing copy, what is your #1 priority?Answer Provided: ? Converting multiple target audiences Correct Answer: Writing for clarity, not cleverness Comments Entered: NA 2. Which BEST describes why you should always define a “Test Hypothesis”Answer Provided: ? To ensure youContinue reading “Results for jur.schouten@gmail.com”

Results for jur.schouten@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which of the following is not a popular prioritization framework?Answer Provided: ? SWOT Correct Answer: SWOT Comments Entered: NA 2. Which is a benefit of testing extreme changes?Answer Provided: ? Generating stronger signals to measure Correct Answer: Generating stronger signals toContinue reading “Results for jur.schouten@gmail.com”

Results for fernando.de.mello5@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which of the following is not a popular prioritization framework?Answer Provided: ? SWOT Correct Answer: SWOT Comments Entered: NA 2. “AIDA” stands for which of the following?Answer Provided: ? Attention, Interest, Desire, Action Correct Answer: Attention, Interest, Desire, Action Comments Entered:Continue reading “Results for fernando.de.mello5@gmail.com”

Results for joanalimasimoes@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. What is a “Bonferroni Correction”?Answer Provided: ? A way to control for false positives when analyzing multiple metrics Correct Answer: A way to control for false positives when analyzing multiple metrics Comments Entered: NA 2. Which of the following is TRUEContinue reading “Results for joanalimasimoes@gmail.com”

Results for carol.vignoli@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which BEST describes an experiment “Hypothesis”?Answer Provided: ? An educated guess as to what will happen during your experiment Correct Answer: An educated guess as to what will happen during your experiment Comments Entered: NA 2. Which of the following bestContinue reading “Results for carol.vignoli@gmail.com”

Results for charithcherry1100@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which of the following is true about p-values?Answer Provided: ? The p-value is the probability of obtaining a result equal to or more extreme than what was observed, assuming that the Null hypothesis is true. Correct Answer: The p-value isContinue reading “Results for charithcherry1100@gmail.com”

Results for shrutichauhancic@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. When collecting test ideas, which of the following should NOT be done?Answer Provided: ? Only consider best practices Correct Answer: Only consider best practices Comments Entered: NA 2. What is a false negative?Answer Provided: ? It is when you don't detectContinue reading “Results for shrutichauhancic@gmail.com”

Results for bhupeshchandrawanshi@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. “AIDA” stands for which of the following?Answer Provided: ? Attention, Interest, Desire, Action Correct Answer: Attention, Interest, Desire, Action Comments Entered: NA 2. Which of the following describes a “Holdout Group”?Answer Provided: ? A control group that does not get exposedContinue reading “Results for bhupeshchandrawanshi@gmail.com”

Results for bhupeshchandrawanshi@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. What is not a valid CRO option when a site has little traffic?Answer Provided: ? Believe you cannot run any tests Correct Answer: Believe you cannot run any tests Comments Entered: NA 2. “PMF” stands for which of the following?Answer Provided:Continue reading “Results for bhupeshchandrawanshi@gmail.com”

Exam answers for: Bhupesh Kumar Chandrawanshi for July 12 2022 11:29:01

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. "AIDA" stands for which of the following?Answer Provided: ? Attention, Interest, Desire, Action Correct Answer: Attention, Interest, Desire, Action Comments Entered: NA 2. Which of the following describes a "Holdout Group"?Answer Provided: ? A control group that does not getContinue reading “Exam answers for: Bhupesh Kumar Chandrawanshi for July 12 2022 11:29:01”

Results for bhupeshchandrawanshi@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. In A/B testing, a “control” is generally which of the following?Answer Provided: ? The version where no changes have been made Correct Answer: The version where no changes have been made Comments Entered: NA 2. What should you do before youContinue reading “Results for bhupeshchandrawanshi@gmail.com”

Results for None

If you’d like to score higher on your next attempt, consider checking out these resources. 1. “PMF” stands for which of the following?Answer Provided: ? Product-Market Fit Correct Answer: Product-Market Fit Comments Entered: NA 2. Which BEST describes why you should always define a “Test Hypothesis”Answer Provided: ? To ensure you learn, no matter what the testContinue reading “Results for None”

Results for mohsend89@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which of the following is not a popular prioritization framework?Answer Provided: ? SWOT Correct Answer: SWOT Comments Entered: NA 2. Which of the following CANNOT help you write copy for conversion?Answer Provided: ? Rewriting competitors' messaging Correct Answer: Mining customer reviewsContinue reading “Results for mohsend89@gmail.com”

Results for mohsend89@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which BEST describes why you should always define a “Test Hypothesis”Answer Provided: ? To ensure you learn, no matter what the test outcome because the experiment will either prove or disprove your hypothesis Correct Answer: To ensure you learn, noContinue reading “Results for mohsend89@gmail.com”

Results for deepak.grtiet@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. An experiment demonstrates a 10% decrease in conversion rate at a 99% confidence level. In what way is this still a ‘win’?Answer Provided: ? Statistically significant results are better than 'flat' results Correct Answer: You’ve saved the business time andContinue reading “Results for deepak.grtiet@gmail.com”

Exam answers for: Fabricio Migues for July 10 2022 12:20:21

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Bayesian and Frequentist are which of the following?Answer Provided: ? Schools of thought in statistics Correct Answer: Schools of thought in statistics Comments Entered: NA 2. Which is a danger of following best practices?Answer Provided: ? They may not workContinue reading “Exam answers for: Fabricio Migues for July 10 2022 12:20:21”

Exam answers for: John Fava for July 10 2022 12:20:30

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is a safe increase in conversion rate that one can promise to a client/stakeholder?Answer Provided: ? You should never promise any outcomes Correct Answer: You should never promise any outcomes Comments Entered: NA 2. Which BEST describes an experimentContinue reading “Exam answers for: John Fava for July 10 2022 12:20:30”

Exam answers for: Dorian Sayazi Crespo Gonzalez for July 10 2022 12:20:47

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which is a danger of following best practices?Answer Provided: ? They may not work for you Correct Answer: They may not work for you Comments Entered: NA 2. What is the most important step you can take to increase yourContinue reading “Exam answers for: Dorian Sayazi Crespo Gonzalez for July 10 2022 12:20:47”

Results for bhupeshchandrawanshi@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which BEST describes a CTA / call to action?Answer Provided: ? A prompt telling the user to take an action Correct Answer: A prompt telling the user to take an action Comments Entered: NA 2. In Frequentist statistics, which of theContinue reading “Results for bhupeshchandrawanshi@gmail.com”

Results for bhupeshchandrawanshi@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which of the following is not a popular prioritization framework?Answer Provided: ? SWOT Correct Answer: SWOT Comments Entered: NA 2. Which is NOT a valid reason to run an A/A test?Answer Provided: ? To demonstrate the problem with “peeking” Correct Answer:Continue reading “Results for bhupeshchandrawanshi@gmail.com”

Results for charithcherry1100@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. What is a safe increase in conversion rate that one can promise to a client/stakeholder?Answer Provided: ? You should never promise any outcomes Correct Answer: You should never promise any outcomes Comments Entered: NA 2. Which of the following characterizes ValueContinue reading “Results for charithcherry1100@gmail.com”

Results for vasisthkumar82@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. “AIDA” stands for which of the following?Answer Provided: ? Attention, Interest, Desire, Action Correct Answer: Attention, Interest, Desire, Action Comments Entered: NA 2. Which of the following best describes why letting a test run longer than planned may not be beneficial?AnswerContinue reading “Results for vasisthkumar82@gmail.com”

Results for vasisthkumar82@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Which is NOT a characteristic of “Novelty effects”?Answer Provided: ? Results seen early on start to fade Correct Answer: New features always perform the best Comments Entered: NA 2. What is a “hero image”?Answer Provided: ? It is the largest imageContinue reading “Results for vasisthkumar82@gmail.com”

Results for vasisthkumar82@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. What is the most important step you can take to increase your funnel’s final conversion rate?Answer Provided: ? Identify the biggest drop-off points and prioritize fixing those Correct Answer: Identify the biggest drop-off points and prioritize fixing those Comments Entered:Continue reading “Results for vasisthkumar82@gmail.com”

Results for fabricio.migues@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. Bayesian and Frequentist are which of the following?Answer Provided: ? Schools of thought in statistics Correct Answer: Schools of thought in statistics Comments Entered: NA 2. Which is a danger of following best practices?Answer Provided: ? They may not work forContinue reading “Results for fabricio.migues@gmail.com”

Results for john_fava@vanguard.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. What is a safe increase in conversion rate that one can promise to a client/stakeholder?Answer Provided: ? You should never promise any outcomes Correct Answer: You should never promise any outcomes Comments Entered: NA 2. Which BEST describes an experiment “Hypothesis”?AnswerContinue reading “Results for john_fava@vanguard.com”

Results for fabricio.migues@gmail.com

If you’d like to score higher on your next attempt, consider checking out these resources. 1. If there is no meaningful difference between your control and variation in terms of your success metric, which of the following should you decide?Answer Provided: ? The test was inconclusive Correct Answer: It depends on the hypothesis Comments Entered: NAContinue reading “Results for fabricio.migues@gmail.com”

Exam answers for: Kathrine Zøllner for June 30 2022 10:51:39

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is qualitative research used for ?Answer Provided: ? Understanding why your users are behaving a certain way Correct Answer: Understanding why your users are behaving a certain way Comments Entered: NA 2. When writing copy, what is your #1Continue reading “Exam answers for: Kathrine Zøllner for June 30 2022 10:51:39”

Exam answers for: Oldrich Kejik for June 29 2022 00:06:50

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which generally happens when testing many variants running at once? Answer Provided: ? The power of your experiment decreases drastically Correct Answer: The power of your experiment decreases drastically Comments Entered: NA 2. Which BEST describes a CTA / callContinue reading “Exam answers for: Oldrich Kejik for June 29 2022 00:06:50”

Exam answers for: Steven Hudspith for June 29 2022 12:50:03

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which BEST describes experiment “Interaction Effects” ? Answer Provided: ? When one experiment impacts the outcome of another Correct Answer: When one experiment impacts the outcome of another Comments Entered: NA 2. Which of the following is not one ofContinue reading “Exam answers for: Steven Hudspith for June 29 2022 12:50:03”

Exam answers for: Mike Harper for June 24 2022 11:29:44

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. In A/B testing, a “control” is generally which of the following? Answer Provided: ? The version where no changes have been made Correct Answer: The version where no changes have been made Comments Entered: NA 2. What is a safeContinue reading “Exam answers for: Mike Harper for June 24 2022 11:29:44”

Exam answers for: Francesca Magana for June 24 2022 14:57:21

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is qualitative research used for ? Answer Provided: ? Understanding why your users are behaving a certain way Correct Answer: Understanding why your users are behaving a certain way Comments Entered: NA 2. What is Peeking? Answer Provided: ?Continue reading “Exam answers for: Francesca Magana for June 24 2022 14:57:21”

Exam answers for: Levon Galstyan for June 24 2022 15:50:23

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is Peeking? Answer Provided: ? Checking to see if you’ve hit sample size before the test is complete with the intention of ending the test. Correct Answer: Looking at test results before the test is complete with theContinue reading “Exam answers for: Levon Galstyan for June 24 2022 15:50:23”

Exam answers for: Stane Zabot for June 23 2022 11:19:50

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is quantitative research NOT used for? Answer Provided: ? Understanding why users are behaving a certain way Correct Answer: Understanding why users are behaving a certain way Comments Entered: NA 2. Which of the following best describes why lettingContinue reading “Exam answers for: Stane Zabot for June 23 2022 11:19:50”

Exam answers for: Michael Johnstone for June 23 2022 12:00:27

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which of the following is an example of scarcity? Answer Provided: ? Only 7 left in stock. Correct Answer: Only 7 left in stock. Comments Entered: NA 2. What should you do if you want to experiment on a websiteContinue reading “Exam answers for: Michael Johnstone for June 23 2022 12:00:27”

Exam answers for: Craig Kistler for June 23 2022 13:16:40

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which of the following is NOT a good strategy for analyzing test results? Answer Provided: ? None of the answers Correct Answer: None of the answers Comments Entered: NA 2. What does “ARPU” stand for? Answer Provided: ? Average RevenueContinue reading “Exam answers for: Craig Kistler for June 23 2022 13:16:40”

Exam answers for: Sander Huisman for June 23 2022 14:29:09

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which of the following describes a “Holdout Group”? Answer Provided: ? A control group that does not get exposed to a treatment/variation Correct Answer: A control group that does not get exposed to a treatment/variation Comments Entered: NA 2. WhichContinue reading “Exam answers for: Sander Huisman for June 23 2022 14:29:09”

Exam answers for: Gabriel Amzallag for June 23 2022 22:34:16

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which BEST defines “Baseline Conversion”? Answer Provided: ? The most recently recorded conversion rate of the metric you want to measure Correct Answer: The most recently recorded conversion rate of the metric you want to measure Comments Entered: NAContinue reading “Exam answers for: Gabriel Amzallag for June 23 2022 22:34:16”

Exam answers for: Paul Barden for June 22 2022 21:55:39

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which is a benefit of testing extreme changes? Answer Provided: ? Generating stronger signals to measure Correct Answer: Generating stronger signals to measure Comments Entered: NA 2. Which of the following is TRUE of best practices? Answer Provided: ? TheyContinue reading “Exam answers for: Paul Barden for June 22 2022 21:55:39”

Exam answers for: Jacob Hartung for June 20 2022 11:14:21

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which BEST defines “Baseline Conversion”? Answer Provided: ? The most recently recorded conversion rate of the metric you want to measure Correct Answer: The most recently recorded conversion rate of the metric you want to measure Comments Entered: NAContinue reading “Exam answers for: Jacob Hartung for June 20 2022 11:14:21”

Exam answers for: Iva Obrovac for June 17 2022 14:39:55

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. “PMF” stands for which of the following? Answer Provided: ? Product-Market Fit Correct Answer: Product-Market Fit Comments Entered: NA 2. If a program’s ratio of “successful” tests (aka “win rate”) to launched tests is constantly 50%, what should you suspect?Continue reading “Exam answers for: Iva Obrovac for June 17 2022 14:39:55”

Exam answers for: Deborah O’Malley for June 16 2022 02:40:04

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What does “ARPU” stand for? Answer Provided: ? Average Revenue Per User Correct Answer: Average Revenue Per User Comments Entered: NA 2. What does “MVT” stand for? Answer Provided: ? Multivariate Test Correct Answer: Multivariate Test Comments Entered: NA 3. WhichContinue reading “Exam answers for: Deborah O’Malley for June 16 2022 02:40:04”

Exam answers for: Nicolai Høg for June 15 2022 13:52:17

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What can we learn from tests run on sites that aren’t our own? Answer Provided: ? What worked for them and their audience Correct Answer: What worked for them and their audience Comments Entered: NA 2. Which of the followingContinue reading “Exam answers for: Nicolai Høg for June 15 2022 13:52:17”

Exam answers for: Cameron Robbins for June 15 2022 14:54:38

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which is NOT a factor in calculating sample size (in Frequentist statistics)? Answer Provided: ? Time Correct Answer: Time Comments Entered: NA 2. Which of the following does NOT describe what an “Experiment” is? Answer Provided: ? A way toContinue reading “Exam answers for: Cameron Robbins for June 15 2022 14:54:38”

Exam answers for: Matheus Rocha for June 15 2022 19:35:01

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which of the following is not one of Cialdini’s 6 Principles of Persuasion? Answer Provided: ? Humour Correct Answer: Humour Comments Entered: NA 2. A test has a >50% lift to your primary metric. What is the first thing youContinue reading “Exam answers for: Matheus Rocha for June 15 2022 19:35:01”

Exam answers for: Fábio Roberto Borges for June 14 2022 17:01:18

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What should you do before you run an experiment? Answer Provided: ? All answers are correct Correct Answer: All answers are correct Comments Entered: NA 2. What does “ARPU” stand for? Answer Provided: ? Average Revenue Per User Correct Answer:Continue reading “Exam answers for: Fábio Roberto Borges for June 14 2022 17:01:18”

Exam answers for: Izabella Bueno for June 14 2022 14:42:35

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What does “KPI” stand for? Answer Provided: ? Key Performance Indicator Correct Answer: Key Performance Indicator Comments Entered: NA 2. Which of the following is NOT a good strategy for analyzing test results? Answer Provided: ? None of the answersContinue reading “Exam answers for: Izabella Bueno for June 14 2022 14:42:35”

Exam answers for: Charles Kim for June 13 2022 19:05:23

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is the most important step you can take to increase your funnel’s final conversion rate? Answer Provided: ? Identify the biggest drop-off points and prioritize fixing those Correct Answer: Identify the biggest drop-off points and prioritize fixing thoseContinue reading “Exam answers for: Charles Kim for June 13 2022 19:05:23”

Exam answers for: Arjan Singh for June 12 2022 13:12:47

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is qualitative research used for ? Answer Provided: ? Understanding why your users are behaving a certain way Correct Answer: Understanding why your users are behaving a certain way Comments Entered: NA 2. Which is NOT a benefit ofContinue reading “Exam answers for: Arjan Singh for June 12 2022 13:12:47”

Exam answers for: Davi Alcantara Nunes for June 11 2022 03:29:00

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which of the following sources of data has the least amount of bias? Answer Provided: ? Meta-analyses Correct Answer: Meta-analyses Comments Entered: NA 2. What is a Sample Ratio Mismatch? Answer Provided: ? The observed traffic split is statistically significantlyContinue reading “Exam answers for: Davi Alcantara Nunes for June 11 2022 03:29:00”

Exam answers for: Matic Kasnik for June 11 2022 03:29:14

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is Peeking? Answer Provided: ? Looking at test results before the test is complete with the intention of ending the test. Correct Answer: Looking at test results before the test is complete with the intention of ending theContinue reading “Exam answers for: Matic Kasnik for June 11 2022 03:29:14”

Exam answers for: Arthur Ribeiro for June 11 2022 03:29:23

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which of the following sources of data has the least amount of bias? Answer Provided: ? Meta-analyses Correct Answer: Meta-analyses Comments Entered: NA 2. Which of the following is a reason to document experiments? Answer Provided: ? All the answersContinue reading “Exam answers for: Arthur Ribeiro for June 11 2022 03:29:23”

Exam answers for: Shiva Manjunath for June 11 2022 03:29:34

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which generally happens when testing many variants running at once? Answer Provided: ? The power of your experiment decreases drastically Correct Answer: The power of your experiment decreases drastically Comments Entered: NA 2. An experiment demonstrates a 10% decrease inContinue reading “Exam answers for: Shiva Manjunath for June 11 2022 03:29:34”

Exam answers for: Storm Jarvie for June 11 2022 10:36:23

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. In A/B testing, a “control” is generally which of the following? Answer Provided: ? The version where no changes have been made Correct Answer: The version where no changes have been made Comments Entered: NA 2. Which of the followingContinue reading “Exam answers for: Storm Jarvie for June 11 2022 10:36:23”

Exam answers for: Joyca Kuijpers for June 11 2022 16:26:10

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. In Frequentist statistics, which of the following is the risk of testing too many KPIs? Answer Provided: ? Increased chance of a false positive Correct Answer: Increased chance of a false positive Comments Entered: NA 2. Which is a dangerContinue reading “Exam answers for: Joyca Kuijpers for June 11 2022 16:26:10”

Exam answers for: Adrian Timar for June 11 2022 03:28:30

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is Peeking? Answer Provided: ? Looking at test results before the test is complete with the intention of ending the test. Correct Answer: Looking at test results before the test is complete with the intention of ending theContinue reading “Exam answers for: Adrian Timar for June 11 2022 03:28:30”

Exam answers for: Mario Breser for June 11 2022 03:28:49

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. Which of the following sources of data has the least amount of bias? Answer Provided: ? Randomized Control Trial results Correct Answer: Meta-analyses Comments Entered: NA 2. Which is NOT a benefit of having “North Star Metric”? Answer Provided: ?Continue reading “Exam answers for: Mario Breser for June 11 2022 03:28:49”

Exam answers for: Raluca Tanase for June 08 2022 10:49:11

If you’d like to score higher on your next attempt, consider checking out these resources. =======   1. Which of the following sources of data has the least amount of bias?Answer Provided: ? Randomized Control Trial results Correct Answer: Meta-analyses Comments Entered: NA   2. Which of the following is not a popular prioritization framework?Answer Provided: ?Continue reading “Exam answers for: Raluca Tanase for June 08 2022 10:49:11”

Exam answers for: Mike Tildsley for June 08 2022 10:49:40

If you’d like to score higher on your next attempt, consider checking out these resources. =======   1. Which of the following sources of data has the least amount of bias?Answer Provided: ? Meta-analyses Correct Answer: Meta-analyses Comments Entered: NA   2. What is a Sample Ratio Mismatch?Answer Provided: ? The observed traffic split is statistically significantlyContinue reading “Exam answers for: Mike Tildsley for June 08 2022 10:49:40”

Exam answers for: Michael McCann for June 07 2022 14:02:31

If you’d like to score higher on your next attempt, consider checking out these resources. =======   1. Which of the following is TRUE of best practices?Answer Provided: ? They are a good starting point for ideation and testing Correct Answer: They are a good starting point for ideation and testing Comments Entered: NA   2. WhichContinue reading “Exam answers for: Michael McCann for June 07 2022 14:02:31”

Exam answers for: Niklas Bräutigam for June 01 2022 15:04:05

If you’d like to score higher on your next attempt, consider checking out these resources. =======   1. Heatmaps are what form of data?Answer Provided: ? Quantitative and Qualitative Correct Answer: Quantitative and Qualitative Comments Entered: NA   2. What is a “hero image”?Answer Provided: ? It is the largest image near the top of a webpageContinue reading “Exam answers for: Niklas Bräutigam for June 01 2022 15:04:05”

Exam answers for: Luísa Campos for June 01 2022 21:25:06

If you’d like to score higher on your next attempt, consider checking out these resources. =======   1. What is Peeking?Answer Provided: ? Looking at test results before the test is complete with the intention of ending the test. Correct Answer: Looking at test results before the test is complete with the intention of ending theContinue reading “Exam answers for: Luísa Campos for June 01 2022 21:25:06”

Exam answers for: for June 01 2022 13:46:09

If you’d like to score higher on your next attempt, consider checking out these resources.   1. What is a Sample Ratio Mismatch?Answer Provided: ? The observed traffic split is statistically significantly different from what was expected Correct Answer: The observed traffic split is statistically significantly different from what was expected Comments Entered: NA   2. WhichContinue reading “Exam answers for: for June 01 2022 13:46:09”

Exam answers for: Bas Buiting for June 01 2022 13:45:51

If you’d like to score higher on your next attempt, consider checking out these resources. =======   1. What is Peeking?Answer Provided: ? Looking at test results before the test is complete with the intention of ending the test. Correct Answer: Looking at test results before the test is complete with the intention of ending theContinue reading “Exam answers for: Bas Buiting for June 01 2022 13:45:51”

Exam answers for: Tobias Feldmann-Wüstefeld for May 31 2022 11:32:26

If you’d like to score higher on your next attempt, consider checking out these resources. ======= 1. What is Peeking?Answer Provided: ? Looking at test results before the test is complete with the intention of ending the test. Correct Answer: Looking at test results before the test is complete with the intention of ending the test.Continue reading “Exam answers for: Tobias Feldmann-Wüstefeld for May 31 2022 11:32:26”

Exam answers for: Brad Croasdell for May 30 2022 11:22:30

If you’d like to score higher on your next attempt, consider checking out these resources. =======     1. What is a safe increase in conversion rate that one can promise to a client/stakeholder?Answer Provided: ? You should never promise any outcomes   Correct Answer: You should never promise any outcomes Comments Entered: NA   2. WhichContinue reading “Exam answers for: Brad Croasdell for May 30 2022 11:22:30”

Exam answers for: Juanma Varo for May 30 2022 11:25:39

If you’d like to score higher on your next attempt, consider checking out these resources. =======   1. Which of the following describes a “Holdout Group”?Answer Provided: ? A control group that does not get exposed to a treatment/variation Correct Answer: A control group that does not get exposed to a treatment/variation Comments Entered: NA  Continue reading “Exam answers for: Juanma Varo for May 30 2022 11:25:39”

Exam answers for: Edd Saunders for May 18 2022 10:40:08

    If you’d like to score higher on your next attempt, consider checking out these resources. =======     1. An experiment demonstrates a 10% decrease in conversion rate at a 99% confidence level. In what way is this still a ‘win’?Answer Provided: ? You’ve saved the business time and money which they might haveContinue reading “Exam answers for: Edd Saunders for May 18 2022 10:40:08”

Best Practices are a starting point – not an excuse to avoid testing

You hear it often, “follow Best Practices” – where a best practice is something that has been generally accepted as the standard way of doing things to help maintain a certain level of quality. Interestingly, you’ll also often hear CROs arguing about them. But why would CROs, who are dedicated to improving the quality ofContinue reading “Best Practices are a starting point – not an excuse to avoid testing”

How many tests should you run per month?

Collin Crowel recently asked the community, “Is test velocity overrated?“. Where test velocity is the number of tests launched over a period of time. About 70% of respondents said “False” and I’m here to tell you that I’m firmly in the “it depends” camp”. Generally speaking, the goal of any Experimentation program should be toContinue reading “How many tests should you run per month?”

Meta-analysis: Do I have to run my test yet again?

Recently, there’s been a bit of buzz within the experimentation community around the concept of meta-analysis. For those unfamiliar with the concept, in a nutshell, it’s about looking at similar experiments altogether to pull insights. In fact, meta-analysis is actually considered the most trustworthy type of analysis out there. That said, it’s still prone toContinue reading “Meta-analysis: Do I have to run my test yet again?”

Ask the Experiment Nation – Should we build our own Experimentation Platform?

Q: Should we build our own Experimentation Platform? Recently, Davis Treybig posted an article about the gap between the best Experimentation Platforms vs. those the rest of us have. In short, he writes about how Experimentation tooling hasn’t kept up with the industry. This got me thinking. Does this mean you should build your ownContinue reading “Ask the Experiment Nation – Should we build our own Experimentation Platform?”

Ask the Experiment Nation – How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?”

Q: How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?” Petra Lylander asked our Slack community this exact question recently which inspired this post. I have to start by clarifying that just putting something out there and seeing what itContinue reading “Ask the Experiment Nation – How do you respond to someone who says, “Let’s not experiment, it just slows us down. Can’t we just put it on our webpage?””

Ask the Experiment Nation – Can you run tests on sites with little traffic?

Q: Can you run tests on sites with little traffic? In a recent post, Magnet Monster posted an interview with Jeremy Horowitz about what to do when you don’t have enough data. You can check out the post here. Jeremy goes on to say that we aren’t in a science lab and we don’t haveContinue reading “Ask the Experiment Nation – Can you run tests on sites with little traffic?”

Question of the week: When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK?

Q: When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK? We recently posed this question to the Experiment Nation community on LinkedIn. This is what they said. Surprisingly, people can get quite passionate around this topic. As you can imagine, there are 2 camps of thinking: “Duh,Continue reading “Question of the week: When is it OK to run 2 A/B tests simultaneously on a site? Is it ever OK?”

Page 3 of 3
1 2 3