I am having a good amount of trouble trying to set up randomization without replacement on a survey. I have six conditions, and I need every participant to be in two, but to not go through the first one twice. I know this could be completed under one randomizer, but I need every participant to answer the same question block after completing the first condition, so they have to come out of the randomizer to complete that.
I know that people have asked questions related to this in the past, and I have read those question threads and tried to implement the solutions, and although it looks like it is set up correctly, I am running into problems. Currently, it seems as though it is working, in that my test run-throughs have not been giving me the same condition twice. However, the embedded data columns are printing out empty in the exported data, and the flow IDs (which I require) are all messed up, leading me to believe something must be wrong. I need access to either the flow IDs or the embedded data in order to know what ‘condition’ participants were randomized into. I previously had the embedded data set just to ‘Text’, before I got a recommendation to use ‘Text Set,’ but I was having the same problem when I was utilizing ‘Text.’ I have tried a few different organizations, including having the branches set under the randomizer and having one embedded data for each block to mark which condition they had seen in the first randomizer and then use that to prevent them from seeing the same condition the second time. I have included screenshots (in the order of the survey flow) which I hope illustrate the survey flow and the problem with the output that I am receiving.
I am not a Qualtrics expert, but I have tried to watch many tutorials, read the Qualtrics pages, and read the forum questions, and have yet to solve the problem. If anyone has any advice, I would greatly appreciate it, and thank you so much.
Page 1 / 1
I could be not understanding this correctly here, so please be patient and re-explain if I’m wrong - could you randomly assign to show 2 blocks at the same time, “in theory” as long as you select “evenly present” it SHOULD NOT present the same 2 blocks/embedded data twice on a single survey, I would test it, but I think that would work.
It may be worth rearranging your block display logic and embedded data naming so it would be Al = 1, Al = 2, etc… rather than Al1, Al2. Then set an additional piece of embedded data at the very top of your survey just called Al, that way later down in the survey flow they’ll be assigned their Al value. There is also always the Qualtrics answer of turn it off (delete it) and on again (rebuild it) to see if those values appear on the export - I would do so by creating a survey copy to test with.
Hi @carcher20 ,
I have six conditions, and I need every participant to be in two, but to not go through the first one twice. I know this could be completed under one randomizer, but I need every participant to answer the same question block after completing the first condition, so they have to come out of the randomizer to complete that.
Can I check does the two conditions needs to be one in AI and one in Human or any two of the six conditions?
If any two of the six conditions, you should amend the “Randomly present” number to 2 on your randomizer and like what kgillis mentioned, you should enable the “evenly present” on the randomizer.
If it needs to be split to get 1 AI and 1 Human, then perhaps you might have the 3 AI to be under a randomizer (randomly present 1, evenly present) within the main randomizer, similarly for the 3 humans. Also, you can use AI=1, AI=2, AI=3 and Human=1, Human=2, Human=3 instead since there wouldn’t be 2 of the same section (AI/Human).
I didn’t set the embedded data as any type, which is Text by default.
Hi @kgillis thank you ever so much for taking the time to respond! I just want to clarify that you are suggesting that I use only one randomizer, with all the blocks underneath, and ask it to evenly present 2 blocks? If so, I used to have it set up like that, but I need a way for participants to answer a block of questions after each condition (which are the same for both conditions I need them to see), and it seems like that was only possible through utilizing two randomizers with the question block in the middle and the end. If I created groups with each condition and the same question block and used that in the randomizer, I was running into the problem where, on their second time answering the questions, their previous answers were auto-filled, which I couldn’t figure out how to turn off.
I like the idea of making a copy and trying off that one - I may do that. I also like the idea of changing the embedded data for AI = 1...to clarify on that point, with the embedded data at the very top of the survey flow, would that not have any value set to it originally? And then there would be only one embedded data field, with 6 different values? If so, I do like that idea and it would certainly be similar. Thank you so much again for your response and apologies for the need for clarification!
Hi @Chee Heng_SZ Thank you so much for taking the time to respond!! I can’t believe you made your own little version of my survey to illustrate your point - I really appreciate that!!!
The two conditions can be any combination (i.e. Ai/Human, Human/Human, Human/AI). I really like the method of using a randomizer to assign participants to two of the embedded data, but I still have the difficulty I mentioned in my rely to kgillis: I need a way for participants to answer a block of questions after each block (which are the same for both ‘conditions’ I need them to see), and it seems like that was only possible through utilizing two randomizers with the question block in the middle and the end. If I created groups with each condition and the same question block and used that in the randomizer, I was running into the problem where, on their second time answering the questions, their previous answers were auto-filled, which I couldn’t figure out how to turn off. Is there a way to have participants answer a block of questions, after each AI/human block, which are the same questions but don’t autofill their previous answers? I used to have a bunch of duplicated blocks which were the same questions but technically different blocks, so they didn’t do the auto-fill, but it created a huge number of columns in the output which I would then have to integrate for all the different conditions, which would be really inefficient. This is only a small test survey, and I have to carry out a much larger one with more conditions, which would be even more inefficient. Thank you so much again for your assistance!
@carcher20 Sounds like the “additional block” to be presented after each of the conditions is the same set of questions, you could also duplicate the block so that it’s technically 2 sets of questions, that way autofill won’t kick in and then if you want you could field edit under “data & analysis” to manipulate the data fields.
In terms of the embedded data, at the top of the survey it would just pull the field, not the value, and then the randomizer would assign that field value.
*There is another way around all this as well (would only be plausible if each block has only a few questions and depending on your question type(s) but you would set up an advanced randomization on the questions themselves where you randomize what would be in the block, but then always show what would be in the additional block.
@kgillis Hello again. Yes, the additional block is the same both times. I could duplicate it in this manner, but where would you suggest I place these in order to ensure that one question block would follow each condition properly? Where does this get placed in the randomization process?
I have never used advanced randomization before, but it seems as though it only allows randomization within question blocks, and I am not sure how that would work in this case. If I were to place all of my ‘conditions’ within one block (feasible), I would still need a way to present the additional question block after the first condition and after the second condition without presenting the same condition twice, and I am not sure how that would work?
If you set up like this (again depending on your questions/types you would move all the options under the “random subset” and then have it evenly display and the choice count you want displayed. On the left hand side I left “Other, please specify” (this is acting like your block where you ALWAYS want it to show after the other questions). I’ll attach a few previews of the survey to show that they’re randomly and evenly presented, but always have the “other, please specify” on there.
survey 1:
survey 2:
survey 3:
survey 4: *I had to get to a 4th survey to get my repeat ice cream flavors presented.
If you could share your questions/question types I could let you know if the above method is feasible. If not then using randomizer is the survey flow would be the better option. And for the block that’s the same and always displayed I would make a copy of it and place it in the survey flow where you have 2 randomizers, with “BLOCK” being the set of questions that everyone sees after their randomized block. Then after your second randomizer instead of adding BLOCK after each block, you would add the BLOCK(COPY) since it technically needs to be a new set of questions even though the questions remain the same.
If you don’t want to do duplication of question blocks, perhaps a loop & merge may be useful while pasting the 15 combinations from the 3 AI and 3 Human into the field1 and field2 and randomly giving 2 loops out of the 15.
However,
1) you will probably need JavaScript to save the embedded data of field1 and field2 to know which AI or/and Human was chosen or use the loop count to refer the combination for AI or/and Human.
2) you probably will have quite a number of columns generated in the export due to 15 possible combinations.
