PDA

View Full Version : WAPS Test Study Techniques



giggawatt
05-20-2014, 07:58 PM
What are some techniques you guys use to study for your WAPS test? How long out do you start studying? What do you do on your study day off before the test?

sandsjames
05-20-2014, 08:10 PM
What are some techniques you guys use to study for your WAPS test?
Most of the time spent on PFE. One book vs 8 books for the same points seemed to make sense. I preferred PFE Gold test questions, then reviewing the areas I had trouble in.



How long out do you start studying? About 3 months. I found that if I studied any longer I got bored, washed out.


What do you do on your study day off before the test? Absolutely nothing related to the test. As a matter of fact, I wouldn't study at all the week before. I do believe that certain people can be "overly prepared". I found that once I was comfortable with the stuff, continuing to study could do more harm than good.

Measure Man
05-20-2014, 09:59 PM
What are some techniques you guys use to study for your WAPS test?

For the SKT, the single MOST important thing you can do is to devote the day AFTER you test to go through your books and highlight the answers to the test you just completed. This few hours will be the most productive study time you will ever do.

The SKT test retains at least 50% of the questions (major re-write) from year to year and as high as 75% of the questions (minor re-write). Unless you've had a major change in the reference material (new CDCs), the tests alternate between a major and minor re-write every other year. Meaning, if you did nothing else but remember the questions from previous year and manage to get only half the remainder correct, you are looking a score in the range of 75-87...well above average.

I can not emphasize this point enough. In fact, what I used to do is after completing the SKT portion, I would remain in the room and review the test over and over simply trying to memorize the questions so I could immediately go out into the car and highlight as many as I could. I would then continue to review those books through the next few days...I would eventually be able for find more than 90 of the 100 questions. These are solid gold for next year. The following year I would literally spend no more than 2-3 days studying nothing else except those previous questions...that was the whole of my SKT prep...the time/benefit of reading the rest of the CDCs was not cost effective. Not to mention some of the old questions actually get brought back time and time again...I'm positive I've seen at least a dozen question on every SKT test I had ever taken from SSgt through MSgt. If they get good stats on a question, they will keep it forever. Oh yeah, in case you didn't know, the AFOMS keeps stats on every questions of every test...how many high scorers got it right vs how many low scorers got it right.

For the PFE, its a matter of studying the book. I would highlight what I thought were the key facts of each paragraph. I would also make up my own questions on index cards, with the answer on the back and study those, like flashcards...even if you never go back and study them, the act of writing out the question and answer is an excellent study technique in itself...and I would get one of those software programs PFE Gold or whatever and do that. Make sure when you miss the questions you refer back to the book and re-read that paragraph...don't just go by the software because they often use slightly different terminology and the test will be the exact words out of the book!


How long out do you start studying?

I am more of a crammer...start hitting the books about a month prior, but really immerse myself into it for about 2 weeks prior.


What do you do on your study day off before the test?

Cram, Cram, Cram. I did not follow the "get a good night's rest model"...I would pull an all-nighter before the test. I would get myself a 12-pack of Diet Pepsi and one of those big tubs of redvines (brain food) and literally stay up all night going over the index cards and software questions...and referencing the book on the ones I miss. This again is mostly PFE stuff. I could point to several questions each time that I specifically remmber picking up the night prior.

Chief_KO
05-20-2014, 10:08 PM
First a couple of no-brainers: Make sure you have the correct study material, especially for the SKT. For the PFE/USAFSE make sure you study the testable chapters (everything in the chapter..pictures, tables, charts,etc.)
When the new PDG comes out, sit down and go page by page to see what is new and what has been deleted. With each years PFE/USAFSE 100% new (questions must be three years or older to be reused) there is a VERY HIGH PROBABILITY that new test questions will come from the new material.
Take the PDG (PFE for you ol'timers) with you when you "sit on the porcelain throne." Read it everytime and before you know it you've read the entire book. Repeat over and over. A couple months prior to test window, ramp it up. Making an outline is a great tool.

Measure Man
05-20-2014, 10:23 PM
Take the PDG (PFE for you ol'timers)

I think the test is still PFE, but the book is PDG...is that right?

Signed,

Old-timer

sandsjames
05-20-2014, 10:28 PM
I think the test is still PFE, but the book is PDG...is that right?

Signed,

Old-timer

I believe so. Wouldn't make much sense for the test to be called "Professional development guide".

Measure Man
05-20-2014, 10:40 PM
I never really understood why the AF would not let people study together.

I mean, okay, I get it if one person has already tested, sure...but for people that didn't test, they should be able to form a study group. Is the AF worried they will learn the information too well? Isn't that the point?

sandsjames
05-20-2014, 10:48 PM
I never really understood why the AF would not let people study together.

I mean, okay, I get it if one person has already tested, sure...but for people that didn't test, they should be able to form a study group. Is the AF worried they will learn the information too well? Isn't that the point?

Especially since all the things you are tested on are things that people are expected to know about either the military or the job. And, I thought that was one of the reasons they changed it from the PFE Study Guide to the Professional Development Guide. That's almost like saying that you can't have more than one person attend a professional development class because it might compromise the test.

Even more so, I can't understand why people from different career fields aren't allowed to study together. Think of married couples. If my wife was in the military, I'm sure we would have been helping each other, test compromise or not.

Airborne
05-20-2014, 10:53 PM
I never really understood why the AF would not let people study together.

I mean, okay, I get it if one person has already tested, sure...but for people that didn't test, they should be able to form a study group. Is the AF worried they will learn the information too well? Isn't that the point?

The AF has no way of knowing who has tested out of cycle etc so it's easier to just say dont do it. Additionally, the AF is not coming to anyone's house breaking up study groups either so go ahead and do it.

Gonzo432
05-21-2014, 12:07 AM
I never really understood why the AF would not let people study together.

I mean, okay, I get it if one person has already tested, sure...but for people that didn't test, they should be able to form a study group. Is the AF worried they will learn the information too well? Isn't that the point?

It's pretty much mandatory everywhere else (tech school, PME, etc.) but forbidden for promotion. I always wondered about that too.

Gonzo432
05-21-2014, 12:16 AM
Go with whatever works for you, but highlighting as soon as you get out of the test is important and works. "Yellow and blue makes green" works to tell you if the questions were on the test 2 years in a row.

I would study 4 months prior. Read the entire book, then go over the highlights multiple times. Repeat. PFE Gold helped me too, scored 89 and 83 on the USAFSE (or whatever the SMSgt/CMSgt test is called now)using it.

giggawatt
05-21-2014, 08:56 AM
For the SKT, the single MOST important thing you can do is to devote the day AFTER you test to go through your books and highlight the answers to the test you just completed. This few hours will be the most productive study time you will ever do.

The SKT test retains at least 50% of the questions (major re-write) from year to year and as high as 75% of the questions (minor re-write). Unless you've had a major change in the reference material (new CDCs), the tests alternate between a major and minor re-write every other year. Meaning, if you did nothing else but remember the questions from previous year and manage to get only half the remainder correct, you are looking a score in the range of 75-87...well above average.

I can not emphasize this point enough. In fact, what I used to do is after completing the SKT portion, I would remain in the room and review the test over and over simply trying to memorize the questions so I could immediately go out into the car and highlight as many as I could. I would then continue to review those books through the next few days...I would eventually be able for find more than 90 of the 100 questions. These are solid gold for next year. The following year I would literally spend no more than 2-3 days studying nothing else except those previous questions...that was the whole of my SKT prep...the time/benefit of reading the rest of the CDCs was not cost effective. Not to mention some of the old questions actually get brought back time and time again...I'm positive I've seen at least a dozen question on every SKT test I had ever taken from SSgt through MSgt. If they get good stats on a question, they will keep it forever. Oh yeah, in case you didn't know, the AFOMS keeps stats on every questions of every test...how many high scorers got it right vs how many low scorers got it right.

For the PFE, its a matter of studying the book. I would highlight what I thought were the key facts of each paragraph. I would also make up my own questions on index cards, with the answer on the back and study those, like flashcards...even if you never go back and study them, the act of writing out the question and answer is an excellent study technique in itself...and I would get one of those software programs PFE Gold or whatever and do that. Make sure when you miss the questions you refer back to the book and re-read that paragraph...don't just go by the software because they often use slightly different terminology and the test will be the exact words out of the book!



I am more of a crammer...start hitting the books about a month prior, but really immerse myself into it for about 2 weeks prior.



Cram, Cram, Cram. I did not follow the "get a good night's rest model"...I would pull an all-nighter before the test. I would get myself a 12-pack of Diet Pepsi and one of those big tubs of redvines (brain food) and literally stay up all night going over the index cards and software questions...and referencing the book on the ones I miss. This again is mostly PFE stuff. I could point to several questions each time that I specifically remmber picking up the night prior.

This is great information! I didn't know about the stats. This is my 4th time testing for MSgt. Last year I got 76 on PFE and 82 on SKT. Would have made it last year, with a 3 EPR, if they didn't jack up the scores. Without that 3 a few years back, I would have made it even with the raised cutoffs.

I've been doing the highlighting bit but never reviewed for a few days after. I have done the test review at the end trying to remember the questions. And I take my materials in the car with me to highlight after leaving the testing room.

I use a PDG Gold type study material. I take the end of chapter quizzes and then highlight in the book the questions I missed. I also do the flash cards too. At the end of the test, I go back and highlight what I remember with a pink highlighter. It stands out from the blue, green, and yellow. This year I went through the books and made all the pink highlighted info into questions on flash cards and marked them with a pink mark so they stand out from my other flash cards.

After the 1st time testing, I learned that the SKT questions come from those long paragraphs of information with no blue, yellow, or green highlighter. So since then I've reviewed those areas every year, underlining key information(usually numbers) and sometimes making flash cards.

I test tomorrow. I've been hitting it hard for a month. I've been casually studying a couple months prior before I got a test date. I study on my phone when I'm waiting somewhere or on the throne.

Every night I study a hour or two and take mini breaks to reset my brain. I usually have some Scotch or a beer while studying and smoke a cigar. Every 10 questions, I take a sip and every question I get wrong, I take a sip. It's made studying fun.

Today, I'm doing a comprehensive review of all my highlighted material and flash cards along with random PDG quizzes.

Thanks for sharing all your tips. This is my year. I will be on that supplemental selection list.

Chief_KO
05-21-2014, 12:13 PM
One big mistake a lot of folks make is trying to predict what they need to score...aka the WAPSCalculator.
News Flash: Last year's cutoff has ZERO impact on the current year.
I've seen too many set a "target" of 75 (or so) and that is the level to which they study. And of course they get a 75...but the cutoff went up. Study with a mindset of "ace'ng" the test. Will you score a 100%...no and if you do OSI will probably look into it.
It took me 10 years to figure that one out (5x for TSgt / 5x for MSgt).

Oh, and since the PDG is available on mp3 and there are many learning games on line (all FREE), there really is no need to pay for PFE Gold or similar programs. (Sorry for those retired Chiefs who own that business).

Measure Man
05-21-2014, 02:27 PM
One big mistake a lot of folks make is trying to predict what they need to score...aka the WAPSCalculator.
News Flash: Last year's cutoff has ZERO impact on the current year.
I've seen too many set a "target" of 75 (or so) and that is the level to which they study. And of course they get a 75...but the cutoff went up. Study with a mindset of "ace'ng" the test. Will you score a 100%...no and if you do OSI will probably look into it.
It took me 10 years to figure that one out (5x for TSgt / 5x for MSgt).

Oh, and since the PDG is available on mp3 and there are many learning games on line (all FREE), there really is no need to pay for PFE Gold or similar programs. (Sorry for those retired Chiefs who own that business).

I never understood it when people said they were "studying for a 75"...why would you study to get 25 wrong??? Just study as much as you can to do the best you can.

ttribe
05-21-2014, 02:54 PM
It's been years since I've had to test. I have to second the technique of highlighting answers. I did so right after testing, and even in between the tests. I had a PFE the had at least 2 years worth of highlights in it when I was going for Tech. Another thing I did was I bought the software available at the time. Then it was HDEL. It was the best money I ever spent on my career. I was not the type to study daily, or regularly. I got the software for both PFE and SKT and took 4 or five days leave in front of my test date. I sat and took the tests to 100% grade completion and then use the software to write in questions from my highlighted books. I had a personal database of 200 questions from my CDCs. I had another 100 or so from the PFE. The first year I used the software and took leave my PFE went up 25 points and my SKT about 20. I made Tech, and was one of only a couple in my career field that did at McGuire that year. Those were some stingy times for making rank. I used the same thing for Master and made it the second time around, I just missed it on the first go. Those extra TIG TIS points put me over that time. ---

On the other foot I have a good friend that made Master and Senior by getting up at 4am every morning and studying for an hour or so before work. Dude was sick if you ask me, but he made his stripes. He probably could have knocked out Chief pretty easy too, had the AF not been so adamant he be skinny, and fleet of foot.

Zxc
05-21-2014, 05:49 PM
This is what I do. I don't recommend it but it works for me.

- Set a schedule / milestones. When do you want to be done w/ each chapter/phase?
- If you want 80+, start 6 to 8 months out.
- Comb the books for what you want to learn. You won't study everything--much of it you know and you would waste your time to look at it again.
- - Line by line, create notecards/flashcards of each individual sentence or piece of information you wish to retain (I do electronic ones now, build in excel and use Flashcard machine app)
- - Never touch your actual books again

- Build secondary/duplicate notecards for questions dealing w/ numerical answers.
- - Group together unlike questions that share only the common numerical answer (e.g., Number of extra duties imposed by a major, max hard labor w/o confinement, how long a UTM has to enroll trainees in CDCs---all 45 days)


The biggest things for me are not wasting time reviewing stuff I’d already get right, and trying to figure out ways to memorize the stuff that I have a hard time remembering. Numbers kill me, so I give myself a way to link together what I do know to what I can’t remember. What that final method does for me, is if I read a question on CDC enrollment, the number 45 may not come to me, but I may remember that the answer is the same as any number of other questions.

Measure Man
05-21-2014, 06:10 PM
This is great information! I didn't know about the stats. This is my 4th time testing for MSgt. Last year I got 76 on PFE and 82 on SKT. Would have made it last year, with a 3 EPR, if they didn't jack up the scores. Without that 3 a few years back, I would have made it even with the raised cutoffs.

I had the fortunate opportunity to get selected for my AFSC SKT rewrite team as a young MSgt. This was an eye opening experience. There are a couple of actual test-taking tips that I've shared on here before...

I would give a disclaimer that these will only work if you are already pretty well-prepared.

1) There will not be more that 3 consecutive correct answers of the same letter. i.e. If you scan your answer sheet and find 4 consecutive "B" answers...at least one of them is wrong. If you are well-prepared there is a good chance you are pretty sure of 3 of them, change the other one. Again, for this to work, you have to be pretty sure of the other 3...or maybe 2 out of 3, leaving you with a 50-50 chance of changing one of the remaining. If you're not well-prepared and took your best guess on all 4, then don't bother. But this is a hard and fast rule that the psychologists at AFOMS insist on.

2) Every letter answer A, B, C, D will have between 23 and 27 correct answers. So, when you are all done, scan the test and if you have 30 "A" answers...at least 3 of them are wrong. If you are prepared, you are probably pretty sure of 24-26 of them...so, up to you if you want to take your chances on the others. This can be a little risky...it's really a matter of how confident you are in most of your answers enough to know which ones you are sketchy on.

These rules apply to both the PFE and SKT,,, assuming they are still in place. They still worked out last time I tested which was 2005.

I maintain that these two techniques added 3-5% of each test I took once I learned of it.

ChiefB
05-21-2014, 08:33 PM
For the SKT, the single MOST important thing you can do is to devote the day AFTER you test to go through your books and highlight the answers to the test you just completed. This few hours will be the most productive study time you will ever do.

The SKT test retains at least 50% of the questions (major re-write) from year to year and as high as 75% of the questions (minor re-write). Unless you've had a major change in the reference material (new CDCs), the tests alternate between a major and minor re-write every other year. Meaning, if you did nothing else but remember the questions from previous year and manage to get only half the remainder correct, you are looking a score in the range of 75-87...well above average.

I can not emphasize this point enough. In fact, what I used to do is after completing the SKT portion, I would remain in the room and review the test over and over simply trying to memorize the questions so I could immediately go out into the car and highlight as many as I could. I would then continue to review those books through the next few days...I would eventually be able for find more than 90 of the 100 questions. These are solid gold for next year. The following year I would literally spend no more than 2-3 days studying nothing else except those previous questions...that was the whole of my SKT prep...the time/benefit of reading the rest of the CDCs was not cost effective. Not to mention some of the old questions actually get brought back time and time again...I'm positive I've seen at least a dozen question on every SKT test I had ever taken from SSgt through MSgt. If they get good stats on a question, they will keep it forever. Oh yeah, in case you didn't know, the AFOMS keeps stats on every questions of every test...how many high scorers got it right vs how many low scorers got it right.

For the PFE, its a matter of studying the book. I would highlight what I thought were the key facts of each paragraph. I would also make up my own questions on index cards, with the answer on the back and study those, like flashcards...even if you never go back and study them, the act of writing out the question and answer is an excellent study technique in itself...and I would get one of those software programs PFE Gold or whatever and do that. Make sure when you miss the questions you refer back to the book and re-read that paragraph...don't just go by the software because they often use slightly different terminology and the test will be the exact words out of the book!



I am more of a crammer...start hitting the books about a month prior, but really immerse myself into it for about 2 weeks prior.



Cram, Cram, Cram. I did not follow the "get a good night's rest model"...I would pull an all-nighter before the test. I would get myself a 12-pack of Diet Pepsi and one of those big tubs of redvines (brain food) and literally stay up all night going over the index cards and software questions...and referencing the book on the ones I miss. This again is mostly PFE stuff. I could point to several questions each time that I specifically remmber picking up the night prior.

These are just about the best inside tips you will ever see on how to attack promotion exams, legally.

I can add just one note and that is to be determined to max out the possible grades in both exams because they are the main member controlled points possible in the final promotion selection process. All other areas of competition are in concrete by the time you test and because of the moving cutoff driven by selection percentages, test results are the only user weapons then available to up your completive edge.

Well done Measure Man.

Chief_KO
05-21-2014, 09:50 PM
1) There will not be more that 3 consecutive correct answers of the same letter. 2) Every letter answer A, B, C, D will have between 23 and 27 correct answers.

I was on a PFE team about 4 years ago and yes, this is absolutely correct. Also, since the PFE/USAFSE is based on one book (PDG) which is digital, you can guarantee (99.999%) that each and every question & correct answer will appear in the book. We would do a word search to confirm this. IF a term appeared more than once and had two different meanings, then then question we were writing was invalid. Best advice to ensure the PDG is updated is to report specific problems or errors to the OPR. Info should be in the front of the book. The issue regarding the PDG is this...AFOMS (or whatever the new name) does not write the material per se. The edit it for readability, etc. but the facts come from various OPRs throughout the AF. The OPR maybe an O5, E6, or GS11 (examples). The tasker goes out for the OPRs to review their section of the PDG for technical accuracy. The OPR may do a cursory look, or may actually overhaul their section...AFOMS does not know. BUT when an Airman identifies a specific error, AFOMS will then track that error with the OPR until it is corrected.

Regarding test challenges (at least to the PDG). As part of the test team process we reviewed all the test challenges to the particular PFE/USAFSE we were writing. Thanks to the digital PDG, the number of valid challenges is <1 per test for the last few years. The typical challenge to the PDG is someone challenging a question based on their personal experience, NOT what is in the test study material. Ex: The PDG states the best method to brew coffee is a drip percolator. An Airman (who works at Starbucks) challenges stating that the best method is a French Press. PFE question is valid based upon the test study material.

Much effort goes into writing the PFE/USAFSE, we spent 6 hours on one question to get it right. The test is much more reliable than in the old days, no more paragraphs with a "choose the best answer". There is one and only one CORRECT answer.

LogDog
05-22-2014, 01:00 AM
I was on a PFE team about 4 years ago and yes, this is absolutely correct. Also, since the PFE/USAFSE is based on one book (PDG) which is digital, you can guarantee (99.999%) that each and every question & correct answer will appear in the book. We would do a word search to confirm this. IF a term appeared more than once and had two different meanings, then then question we were writing was invalid. Best advice to ensure the PDG is updated is to report specific problems or errors to the OPR. Info should be in the front of the book. The issue regarding the PDG is this...AFOMS (or whatever the new name) does not write the material per se. The edit it for readability, etc. but the facts come from various OPRs throughout the AF. The OPR maybe an O5, E6, or GS11 (examples). The tasker goes out for the OPRs to review their section of the PDG for technical accuracy. The OPR may do a cursory look, or may actually overhaul their section...AFOMS does not know. BUT when an Airman identifies a specific error, AFOMS will then track that error with the OPR until it is corrected.

Regarding test challenges (at least to the PDG). As part of the test team process we reviewed all the test challenges to the particular PFE/USAFSE we were writing. Thanks to the digital PDG, the number of valid challenges is <1 per test for the last few years. The typical challenge to the PDG is someone challenging a question based on their personal experience, NOT what is in the test study material. Ex: The PDG states the best method to brew coffee is a drip percolator. An Airman (who works at Starbucks) challenges stating that the best method is a French Press. PFE question is valid based upon the test study material.

Much effort goes into writing the PFE/USAFSE, we spent 6 hours on one question to get it right. The test is much more reliable than in the old days, no more paragraphs with a "choose the best answer". There is one and only one CORRECT answer.
Your information and experience sounds similar to my experience when I help with a major re-write of our SKT test. Our CDC writer, myself and two other MSgts had two manuals, about 24 sections in each, in which to draw questions from. I had a two-page section to write a question and it took me most of the day to come up with a relevant question. On one section dealing with computer administration operations one of the MSgts was going for questions that maybe one in a hundred people in the career field could answer. We convinced him to drop the questions and to keep it simple because they were designed for people who worked the Data Design Center at Gunter AS.

What we agreed upon as a group was not to make a the hardest test we could but one that tested the knowledge of what the people in the career field were doing and what they should already know through experience or reading (studying) about in the manuals.

DWWSWWD
05-22-2014, 02:16 AM
Absolutely nothing related to the test. As a matter of fact, I wouldn't study at all the week before. I do believe that certain people can be "overly prepared". I found that once I was comfortable with the stuff, continuing to study could do more harm than good. Impossible in my experience to be too prepared for a test. I start 6 months out and get at least an hour a day, weekends off. I read the books cover to cover twice before I started any test prep stuff. I have never studied and failed to get promoted.

coloringoutsidethelines
05-22-2014, 04:09 AM
A lot of great advice and I only have this one thing to add. I take at least a week off before the test and I treat my study time just like a work day. I get up at the same time and keep the breaks and distractions down. I put my nine hours in and it seems to stick.

Measure Man
05-22-2014, 04:17 AM
A lot of great advice and I only have this one thing to add. I take at least a week off before the test and I treat my study time just like a work day. I get up at the same time and keep the breaks and distractions down. I put my nine hours in and it seems to stick.

That is probably effective...but, for me, I have always been morally and fundamentally opposed to using leave for advancing my career. To me that time belongs to the opposite...self and family. In the grand scheme of life, more than enough time is spent on career. I know it probably works out the same as spending off-duty hours studying, just spending leave time studying has always rubbed me the wrong way. I never did it.

sandsjames
05-22-2014, 10:31 AM
Impossible in my experience to be too prepared for a test. I start 6 months out and get at least an hour a day, weekends off. I read the books cover to cover twice before I started any test prep stuff. I have never studied and failed to get promoted.

Of course it's different techniques for different people. I see the difference in my students. There are some who do well by studying a lot. Then I've always got 1 or 2 who can answer all the questions in the classroom, prepare very well, and when it comes to the test they will overthink the answers and usually talk themselves out of the right answer because it seems "too obvious". These are usually the guys who are studying up to the last second.

Chief_KO
05-22-2014, 12:17 PM
If you're a SNCO and are offered the opportunity to be on an SKT team, jump on it. As said before it is a great experience and you'll really understand "the process" of how everything fits together into a pretty damn good system.

The SKT is a lot harder to write than the PFE/USAFSE due to the multiple books, etc. When we arrived at Randolph, there was an SKT team in the room across from us. They were still there when we were done and heading home.

sandsjames
05-22-2014, 12:41 PM
If you're a SNCO and are offered the opportunity to be on an SKT team, jump on it. As said before it is a great experience and you'll really understand "the process" of how everything fits together into a pretty damn good system.

The SKT is a lot harder to write than the PFE/USAFSE due to the multiple books, etc. When we arrived at Randolph, there was an SKT team in the room across from us. They were still there when we were done and heading home.

To go along with this, I think that everyone (NCO/SNCO) should have to be involved in at least one U&TW during their career. It really clarifies how the STS is created/revised and why certain tasks are listed as they are.

In addition to this, when studying, ensure that you take a look at the STS and the proficiency codes. Understand those proficiency codes because that is how the questions are formed. For instance, if something is a "b" in the proficiency code, you just have to know the basic steps for the process. If it's a "c" then you need to understand the steps much better. Same goes for the knowledge level (uppercase). Know the difference between task items and knowledge items. It can save a lot of wasted time. For instance, you have a "B" proficiency code, you don't need to know step by step processes, you will just need to be able to identify facts. This can save a lot of time when it comes to memorizing facts vs. memorizing steps.

giggawatt
05-22-2014, 02:04 PM
If you're a SNCO and are offered the opportunity to be on an SKT team, jump on it. As said before it is a great experience and you'll really understand "the process" of how everything fits together into a pretty damn good system.

The SKT is a lot harder to write than the PFE/USAFSE due to the multiple books, etc. When we arrived at Randolph, there was an SKT team in the room across from us. They were still there when we were done and heading home.

What is with the questions on both the PFE and SKT?

Example.

Question: What is Xxxxx Yyyyy Zzzz?
A) Lemons only
B) Lemons and limes
C) Apples only
D) Apples and pears

A lot of questions like that. Frustrates the hell out of me.

Measure Man
05-22-2014, 04:02 PM
On one section dealing with computer administration operations one of the MSgts was going for questions that maybe one in a hundred people in the career field could answer. We convinced him to drop the questions and to keep it simple because they were designed for people who worked the Data Design Center at Gunter AS.

What we agreed upon as a group was not to make a the hardest test we could but one that tested the knowledge of what the people in the career field were doing and what they should already know through experience or reading (studying) about in the manuals.

Okay, so for those curious about how the stats work and what makes a "Good question" in the eyes of AFOMS, I can explain this a little.

The two WORST questions are the one that everyone misses and the one hat everyone gets correct. The purpose of the questions is to differentiate, or discriminate, those people who know the material from those who do not. So, ideally, what they want is for the "high scorers" to get a question correct and the "low scorers" to get the question incorrect.

So, they actually get the previous years results, no names, just scores and answers. Depending on the number of test takers they will decide on what percentage is considered a high or low scorer. For example, on a very large population they will take the top 25% and call them the high scorers, and the bottom 25% and call them the low scorers. If it is a smaller AFSC and has a small population, the will actually just split it in half with the top half being high and the bottom half being low.

Okay, so now they look at each question. For simplicity, let's assume there are 10 high scorers and 10 low scorers. Their ideal question would have all 10 high scorers getting the question correct and all 10 low scorers getting it incorrect. I can't recall exactly what the formula was, but they divide the percent of high scores by the percentage of low scores somehow and get a quality factor or something for the question. If the results are upside down...low scorers getting it correct while high scorers miss it, that is a bad question and gets either tossed for the next year, or the new team can edit it.

I think I have to take back my original statment of the worst questions...I think one that all the low scorers get correct while the high scorers miss is actually the worst question, as it inversely differentiates the population.

Among the "good questions"...they will also look at which wrong answer the peoople who missed it chose. If NOBODY chose option C, then it is what they call an implausible distraction. All the wrong answers are called distractions, and they are all supposed to be plausible. They should be plausible enough that some people will choose that answer. That is why you don't get the silly wrong answers that you see in some other tests...all of the choices must seem plausible to someone who doesn't really know their stuff.

As you are writing the questions, they will also insist that all of the answers be approximately the same length...so you can't have like 2 answers are one-word, 1 answer is a sentence, and another one three sentences. If the correct answer is a sentence, then all the incorrect ones should also be a sentence long. If the correct answer is one-word, then all the incorrect ones should also be one or two words.

It has been a little while, so there is a chance I don't have the terminology exactly correct. But, there really is a science to making good tests.

Chief_KO
05-22-2014, 05:42 PM
To go along with this, I think that everyone (NCO/SNCO) should have to be involved in at least one U&TW during their career. It really clarifies how the STS is created/revised and why certain tasks are listed as they are.

In addition to this, when studying, ensure that you take a look at the STS and the proficiency codes. Understand those proficiency codes because that is how the questions are formed. For instance, if something is a "b" in the proficiency code, you just have to know the basic steps for the process. If it's a "c" then you need to understand the steps much better. Same goes for the knowledge level (uppercase). Know the difference between task items and knowledge items. It can save a lot of wasted time. For instance, you have a "B" proficiency code, you don't need to know step by step processes, you will just need to be able to identify facts. This can save a lot of time when it comes to memorizing facts vs. memorizing steps.

100% spot on!!

Chief_KO
05-22-2014, 05:55 PM
What is with the questions on both the PFE and SKT?

Example.

Question: What is Xxxxx Yyyyy Zzzz?
A) Lemons only
B) Lemons and limes
C) Apples only
D) Apples and pears

A lot of questions like that. Frustrates the hell out of me.

Well, it's been 18 years since I've taken an SKT and my last frame of reference to a PFE/USAFSE is 2008 when I was on a writing team but here's how I used to brief it to my Amn:

Let's say you're writing a test on sports. Here's a proposed question:
Who is the greatest basketball player of all time?

a.Kareem Abdul-Jabbar
b.Michael Jordan
c.Magic Johnson
d.Babe Ruth
This question would be no good...too subjective "best of all time" and option d is not a plausable answer.

So here's attempt #2:
Which basketball player won college basketball championships while at UCLA and professional basketball championships with Milwaukee and Los Angeles?

a.Kareem Abdul-Jabbar
b.Michael Jordan
c.Magic Johnson
d.Babe Ruth
Still not good, too much information in the stem (question) that is leading you to a couple of possible choices (Los Angeles) (and option d still not plausable).

Attempt #3:
Which basketball player won 3 college basketball championships and 6 professional basketball championships?

a.Kareem Abdul-Jabbar
b.Magic Johnson
c.Michael Jordan
d.Bill Russell
All four choices are plausible. This would require the test taker to have read the material and be able to piece together the answer (totalling the information contained within the study material). And of course the only CORRECT answer is a.
In this scenario the answers are sorted alphabetically. They can be sorted several different ways (and throughout the test different methods are used) to maintain the even distribution between options a, b, c, d.

The AFOMS test psychologists job is to ensure the questions are clear, concise, and easy to follow.

One other bit of test trivia...the test accuracy rate (% of questions scored) on the PFE/USAFSE (sorry I can't speak of the SKT) exceeds the threshold established by colleges, universities, and industry.

LogDog
05-22-2014, 06:58 PM
Okay, so for those curious about how the stats work and what makes a "Good question" in the eyes of AFOMS, I can explain this a little.

The two WORST questions are the one that everyone misses and the one hat everyone gets correct. The purpose of the questions is to differentiate, or discriminate, those people who know the material from those who do not. So, ideally, what they want is for the "high scorers" to get a question correct and the "low scorers" to get the question incorrect.

So, they actually get the previous years results, no names, just scores and answers. Depending on the number of test takers they will decide on what percentage is considered a high or low scorer. For example, on a very large population they will take the top 25% and call them the high scorers, and the bottom 25% and call them the low scorers. If it is a smaller AFSC and has a small population, the will actually just split it in half with the top half being high and the bottom half being low.

Okay, so now they look at each question. For simplicity, let's assume there are 10 high scorers and 10 low scorers. Their ideal question would have all 10 high scorers getting the question correct and all 10 low scorers getting it incorrect. I can't recall exactly what the formula was, but they divide the percent of high scores by the percentage of low scores somehow and get a quality factor or something for the question. If the results are upside down...low scorers getting it correct while high scorers miss it, that is a bad question and gets either tossed for the next year, or the new team can edit it.

I think I have to take back my original statment of the worst questions...I think one that all the low scorers get correct while the high scorers miss is actually the worst question, as it inversely differentiates the population.

Among the "good questions"...they will also look at which wrong answer the peoople who missed it chose. If NOBODY chose option C, then it is what they call an implausible distraction. All the wrong answers are called distractions, and they are all supposed to be plausible. They should be plausible enough that some people will choose that answer. That is why you don't get the silly wrong answers that you see in some other tests...all of the choices must seem plausible to someone who doesn't really know their stuff.

As you are writing the questions, they will also insist that all of the answers be approximately the same length...so you can't have like 2 answers are one-word, 1 answer is a sentence, and another one three sentences. If the correct answer is a sentence, then all the incorrect ones should also be a sentence long. If the correct answer is one-word, then all the incorrect ones should also be one or two words.

It has been a little while, so there is a chance I don't have the terminology exactly correct. But, there really is a science to making good tests.
When I did the major SKT re-write, we took the previous year's test. Then we were told 50% of the questions were to be tossed and which sections from our two manuals the replacement questions would come from. It was our decision which questions were dropped. What you said about AFOMS may be true but weren't given that information nor was there any directives from the AFOMS prompter (I forget her exact title) as to which questions needed replacing.

The prompter reviewed our questions, checked the references to ensure the answer to the questions were valid, and checked to ensure the questions and answers met AFOMS standards. If they did she passed them on to her supervisor who reviewed them and if they were okay they went for a final review.

As for the format of the answers, you are correct. The answers must not be written that would make it "obvious" but not too difficult to recognize or figure out. If you have a sentence for an answer then all answers for that question need to be approximately the same length.

As you said, test making is a science and it's not like high school or college tests where the instructor makes their own questions and answers. While at the NCO Academy in Germany in 1985, an instructor there earned a college degree by taking CLEP and DANTES tests only. Our instructors told us one of his tricks to figuring out the answers was the longest answer is usually the correct answer because it's harder to write a long, incorrect answer. The AF promotion tests are designed to eliminate the problems civilian test writers have.

LogDog
05-22-2014, 07:02 PM
Well, it's been 18 years since I've taken an SKT and my last frame of reference to a PFE/USAFSE is 2008 when I was on a writing team but here's how I used to brief it to my Amn:

Let's say you're writing a test on sports. Here's a proposed question:
Who is the greatest basketball player of all time?

a.Kareem Abdul-Jabbar
b.Michael Jordan
c.Magic Johnson
d.Babe Ruth
This question would be no good...too subjective "best of all time" and option d is not a plausable answer.

So here's attempt #2:
Which basketball player won college basketball championships while at UCLA and professional basketball championships with Milwaukee and Los Angeles?

a.Kareem Abdul-Jabbar
b.Michael Jordan
c.Magic Johnson
d.Babe Ruth
Still not good, too much information in the stem (question) that is leading you to a couple of possible choices (Los Angeles) (and option d still not plausable).

Attempt #3:
Which basketball player won 3 college basketball championships and 6 professional basketball championships?

a.Kareem Abdul-Jabbar
b.Magic Johnson
c.Michael Jordan
d.Bill Russell
All four choices are plausible. This would require the test taker to have read the material and be able to piece together the answer (totalling the information contained within the study material). And of course the only CORRECT answer is a.
In this scenario the answers are sorted alphabetically. They can be sorted several different ways (and throughout the test different methods are used) to maintain the even distribution between options a, b, c, d.

The AFOMS test psychologists job is to ensure the questions are clear, concise, and easy to follow.

One other bit of test trivia...the test accuracy rate (% of questions scored) on the PFE/USAFSE (sorry I can't speak of the SKT) exceeds the threshold established by colleges, universities, and industry.
I just took an on-line test seeing how much I know of classic literature. Granted, each question had only three answers and some of the answers were obvious but they didn't come anywhere near the standards of AFOMS. I scored 14 out of 25 for 56% but the kicker was I only read three of the books the question were asked. The result shows I was a good guesser.

Chief_KO
05-22-2014, 08:00 PM
The biggest weakness (or single point of failure) to the entire AF training program is the lack of input from the field.
Surveys are used...which are good. The problem lies in the low return rate, or the unusable inputs from the field (all "As", A,B,C,D zigzag pattern, etc.).
Here's the skinny:

Surveys part 1
CRITICAL to every step of your career development and every promotion
Career development
CFETP (3-level CTS / STS) (5-level JQS) (7-level JQS)
5-level, 7-level, 9-level schools
PME (ALS, NCOA, SNCOA, CLC)
CDCs
PDG
Promotion
SKT
PFE
USAFSE

Surveys part 2
Types (every 2 years)
Job Inventory (1-9) (all in AFSC)
How much time do spend performing each task in your present job?
Training Emphasis (1-9) (fair sampling of AFSC)
How much emphasis should be spent training on specific tasks in the 3-level schoolhouse?
Task Learning Difficulty (1-9) (fair sampling of AFSC)
How difficult is it to learn specific tasks?
Testing Importance (<100 SNCOs in AFSC)
What are the important tasks to include in E5, E6, E7 SKTs?
2008: 30 of 77 2E172 SNCOs (39%) responded
Surveys part 3
Military Knowledge & Testing System (MKTS)
Primary instrument to determine which (and how much of each) topics are included in the Professional Development Guide (PDG)
Over 5,000 NCOs are asked to rate each (over 350) topics according to the need for professional knowledge or skill in that topic based on their present rank
The survey is generally constructed a year after the effective date of the most recent edition of the PDG.*
This ensures that each survey respondent has had adequate time to read, and in almost all cases, study the PDG.*
To a large extent, YOU, and many others like you, determine the content of the study guides.
The MKTS Advisory Council Workshop (CMSAF, MAJCOM/DRU/FOA CCCs) review MKTS survey data and finalize the PDG topic outline

Class5Kayaker
05-22-2014, 08:10 PM
As you are writing the questions, they will also insist that all of the answers be approximately the same length...so you can't have like 2 answers are one-word, 1 answer is a sentence, and another one three sentences. If the correct answer is a sentence, then all the incorrect ones should also be a sentence long. If the correct answer is one-word, then all the incorrect ones should also be one or two words.


But I like tests that have answers that are "too long to be wrong!" ;)

Measure Man
05-22-2014, 08:13 PM
When I did the major SKT re-write, we took the previous year's test. Then we were told 50% of the questions were to be tossed and which sections from our two manuals the replacement questions would come from. It was our decision which questions were dropped. What you said about AFOMS may be true but weren't given that information nor was there any directives from the AFOMS prompter (I forget her exact title) as to which questions needed replacing.

Yes, I remember taking the tests first also. I was also there for a major re-write. But, they did give us specific questions that had to be fixed or eliminated due to bad stats.

sandsjames
05-22-2014, 08:17 PM
The biggest weakness (or single point of failure) to the entire AF training program is the lack of input from the field.
Surveys are used...which are good. The problem lies in the low return rate, or the unusable inputs from the field (all "As", A,B,C,D zigzag pattern, etc.).
Here's the skinny:

Surveys part 1
CRITICAL to every step of your career development and every promotion
Career development
CFETP (3-level CTS / STS) (5-level JQS) (7-level JQS)
5-level, 7-level, 9-level schools
PME (ALS, NCOA, SNCOA, CLC)
CDCs
PDG
Promotion
SKT
PFE
USAFSE

Surveys part 2
Types (every 2 years)
Job Inventory (1-9) (all in AFSC)
How much time do spend performing each task in your present job?
Training Emphasis (1-9) (fair sampling of AFSC)
How much emphasis should be spent training on specific tasks in the 3-level schoolhouse?
Task Learning Difficulty (1-9) (fair sampling of AFSC)
How difficult is it to learn specific tasks?
Testing Importance (<100 SNCOs in AFSC)
What are the important tasks to include in E5, E6, E7 SKTs?
2008: 30 of 77 2E172 SNCOs (39%) responded
Surveys part 3
Military Knowledge & Testing System (MKTS)
Primary instrument to determine which (and how much of each) topics are included in the Professional Development Guide (PDG)
Over 5,000 NCOs are asked to rate each (over 350) topics according to the need for professional knowledge or skill in that topic based on their present rank
The survey is generally constructed a year after the effective date of the most recent edition of the PDG.*
This ensures that each survey respondent has had adequate time to read, and in almost all cases, study the PDG.*
To a large extent, YOU, and many others like you, determine the content of the study guides.
The MKTS Advisory Council Workshop (CMSAF, MAJCOM/DRU/FOA CCCs) review MKTS survey data and finalize the PDG topic outlineVery good point about input from the field.

What I've found during a couple U&TWs that I sat in on was that there wasn't nearly enough input from the junior NCOs. Specifically to our career field we had E7s and E8s having the final say on what stayed in. Even though the E5/E6s would make very clear that we no longer worked on certain equipment (or that it was very rare) the SNCOs insisted the tasks were critical.

For instance, we are required to teach several hours of instruction on circuit cards/resistors/etc that relate to our career field but haven't been relevant in the last 10-15 years because when it comes to fixing those items on our generators, all we do is remove and replace an entire circuit card. We don't ever (or very rarely) deal with the actual components on the circuit card. We take out 4 screws, remove the card, and replace it with a new one.

Is it good info to know? Yeah, I suppose. However, should it be part of the 3/5/7 level training? Not at all. Definitely not as in depth as we are required to go into it. But there are senior enlisted still at the U&TWs that actually worked on generators when we actually did solder/desolder these components, so they insist that they stay as critical tasks.

It's unfortunate, and I understand the one base somewhere in the world may do it, but it's not common, and it's not critical.

Chief_KO
05-22-2014, 09:59 PM
Sandsjames,
Sounds like what the career field needs is a forward-thinking, visionary CFM. Back when I was teaching/instructor supervisor/element chief, our AFSC was totally overhauled, merging computer maintenance with crypto maintenance. There was a lot of "fluff" taught in both schoolhouses (old theory that was no longer applicable, or was just "knowledge" that could be in a CDC). It was all yanked out as well as specifying certain pieces of equipment in the STS (CTS). This allowed Keesler to update the course equipment regularly without a major re-write.
A year after the new course was on line, MSgt_KO was leading a group of Chiefs on a tour of the course. One of the old crypto Chiefs lamented that we needed to teach the crypto theory (how the devices scrambled the data). I asked him why, was there test equipment to verify the scrambling? was there an alignment or other maintenance action to correct the scrambling if it was incorrect? Of course the answer to both is no. I remember seeing the other Chiefs smirking when I challenged him...

sandsjames
05-22-2014, 10:13 PM
Sandsjames,
Sounds like what the career field needs is a forward-thinking, visionary CFM. Back when I was teaching/instructor supervisor/element chief, our AFSC was totally overhauled, merging computer maintenance with crypto maintenance. There was a lot of "fluff" taught in both schoolhouses (old theory that was no longer applicable, or was just "knowledge" that could be in a CDC). It was all yanked out as well as specifying certain pieces of equipment in the STS (CTS). This allowed Keesler to update the course equipment regularly without a major re-write.
A year after the new course was on line, MSgt_KO was leading a group of Chiefs on a tour of the course. One of the old crypto Chiefs lamented that we needed to teach the crypto theory (how the devices scrambled the data). I asked him why, was there test equipment to verify the scrambling? was there an alignment or other maintenance action to correct the scrambling if it was incorrect? Of course the answer to both is no. I remember seeing the other Chiefs smirking when I challenged him...

We definitely need an overhaul and the forward thinking you speak of. We are currently working on a course rewrite which I hopefully will be involved with (I was involved in the last one we did while I was active duty), or at least can consult with those who are writing it.

You make a great point about the theory. We are so big on teaching theory. What we need to be focusing on is application. We need to be sending people into the operational side of things with the ability to do their job (at a 3 skill level). I don't care if the troop knows the components of the fuel pump. I care that he knows how to locate the fault and replace the component.

Hopefully as the newer generation takes the lead on these things there will be progress.

Gonzo432
05-23-2014, 01:39 AM
But I like tests that have answers that are "too long to be wrong!" ;)

This old SMSgt I knew at Tyndall (I was a jeep SSgt at the time) said, "Three long and one short, pick the short. Three short and one long, pick the long. All the same length, pick C."