Jack Koziol Posts

Help and Frequently Asked Questions

Why do you allow community contributed content and questions?

Skillset allows community members to edit questions, contribute new questions, rate the quality of questions, grade test items and provide feedback to other users.blooms-taxonomy

When you contribute a question, you are actually testing your own knowledge of a subject. It is impossible to craft a quality test question without deep knowledge of a subject. A wealth of academic research has been done on Scales of Understanding, showing that when you can apply your knowledge in the form of a question, you have mastered a subject.

While you may encounter some poorly crafted questions on Skillset (which are quickly filtered out and deleted), the sum of all of the community contributed content provides an astounding insight into the zeitgeist of a particular high stakes exam. The “Wisdom of Crowds” effect, when properly moderated, ensures that the content for each high stakes exam is always current, topical and of high quality.

So go ahead. Give it a try. Challenge yourself by trying to create a great question. See how it improves your understanding of a topic.


I found a question that is incorrect or not appropriate. What should I do?

Skillset constantly evaluates test question items for quality algorithmically using feedback from our users. Simply answering and viewing a question gives Skillset data to feed our deep learning engine. If we detect that a question is the wrong difficulty for the current level it is in, we move it to the appropriate skill level. If we detect that a question is of low quality, we will delete it and prevent it from being shown to other users.

If you encounter a question you feel is incorrect or inappropriate, feel free to edit the question and fix it. This aids in your understanding of the topic being tested. You can also rate the question, using the star rating, which accelerates the question quality promotion or demotion within the system.

New or poorly rated questions don’t count towards your Exam Readiness, so don’t fret, just move on.


Why should I answer a peer assessment question?

Multiple choice test questions are flawed. They are wildly popular because solely because they are easy to design and can be graded by machines, NOT because they are a better test of knowledge. A multiple choice correct answer can be guessed, gamed, and often knowledge of “how to answer” the question is more important than a deep understanding of the topic being tested on. We have all experienced this in our lives, and there is much written about it. Even though the exam you are preparing for may not require essay or hands-on testing, there is no better test of your own mastery of a subject than to answer a complex item related to a key concept in written form. This can take many forms:

  • A short essay question
  • Analyze a set of data and come to a conclusion
  • Perform a hands-on task correctly and report output

The goal of Skillset is to guarantee you pass your high stakes exam. On the first try. Knowing that you have deep understanding of an important topic is key to passing your exam. Peer assessment questions are important data points for Exam Readiness.

Skillset Pro subscribers have their Peer Assessment questions graded faster and are not blocked from advancing in skills while their Peer Assessment questions are being graded.

If you want to replicate what your exam is like, sans Peer Assessment questions, try out the Simulated Exam.


How do you determine Readiness Level?

Everything you do on Skillset is used as a factor for Readiness Level. Here are the inputs to the Readiness Level calculation:

  • Frequency of usage
  • Reinforcement question
  • Training completed
  • Test item scores
  • Peer Assessment scores
  • Simulated exam scores
  • Test scores
  • Interdependency between proficiencies on related skills
  • Historical candidate data



A morning at the gener8tor mentor swarm

I had an opportunity to participate as a mentor in the Madison, WI gener8tor Mentor Swarm today. If you are not familiar, gener8tor is a Wisconsin startup incubator that has some impressive momentum in the state. Since forming in 2012, gener8tor has graduated 28 start-ups that have raised a total of about $30 million.

The Mentor Swarm
What is it? In summary, the idea is to bombard the founders of the portfolio companies with a week of short meetings with local people that can offer advice. More subtly, I see it was a chance for founders to practice their pitches and get some exposure to the types of questions they will be asked by VCs when they graduate from the program. It also serves as a structured networking event, the mentors included entrepreneurs that have raised a lot of capital and exited, as well a number of partners at various VC firms looking for companies to deploy capital.

gener8tor has 5 companies in this class:

  • GroceryKey, a frictionless last mile solution targeted at independent grocers
  • BrightCellars, a monthly wine club that uses data science to recommend wine for wine enthusiasts
  • Carson Life, line of beauty products that targets the Hispanic market.
  • Passage, ticketing and payments product for speciality events.
  • AltusMedicalGroup, a B2B medical education compliance company

Before the swarm, I thought how can I make myself useful in a short meeting? What can I offer to some early stage startups in an incubator? Instead of trying to fully understand each business model and their associated financial, go to market, team building, and funding plans, I decided to focus on two narrow but critical areas, the core product / value proposition and customer acquisition.  I have had some success in these areas previously, additionally I am always a bit selfishly interested in how smart people solve hard problems.

Product/Value Prop
First, I want to understand the problem the company is solving with their product. Richard Yau at BrightCellars has identified that many early career professionals want to understand the basics of wine selection. They are graduating from cheap beer in a red solo cup to a more “mature” drink. Daniel Guerra at AltusMedical has found that medical centers have a vested interest in getting their health professionals in compliance with their continuing education requirements. Jeremy Neren at GroceryKey has found that the process for enrolling an independent grocery store on existing last mile grocery delivery services is painful and expensive for the grocer.

One thought that struck me is that all of these companies are looking to displace existing products in the market. Most startups do. At this early stage in the product lifecycle, it’s important to remember the Startup to the Power of 10 rule. Namely, that the product has to be 10x better to displace an incumbent solution. If it is just 2 or 3 times better than an existing solution, a customer is likely to go with “the known” verses “the unknown” and stick with what they already have. For example, GroceryKey has to compete against some mighty incumbents – MyWebGrocer, Peapod and ugh.. Amazon. What are the benefits that GroceryKey can bring to the table that could make it 10x better?

  1. Frictionless enrollment
  2. A lovely user experience (both for grocers and shoppers)
  3. Network of happy users
  4. White labeling the front end
  5. Local customers through content marketing

Jeremy has some good ideas related to these subjects. I see #5 as being key. If GroceryKey can bring incremental business into the grocer, the value prop will be hard to resist. This will allow him to build his network and take advantage of the network effect.

Customer Acquisition
All of the companies in the gener8tor class have market proven business models. Everyone had some paying customers and made sure that I knew that. Carson Life has their products distributed through every Walmart in Puerto Rico. BrightCellars has 1000+ sommelier wannabes drinking wine they love each month. GroceryKey has a well known UW campus area grocery store already using the platform. AltusMedical has a Fortune 50 customer on a multi-million dollar contract.

Founders can get their first dollars and customers through sheer will. By doing things that dont scale. Ben Silbermann at Pinterest famously recruited his first users by walking up to random strangers at coffee shops and asking them to use his app. But, where do you go from there? How will these gener8tor startups light a fire that doesnt burn out? Early stage startups need to acquire customers efficiently, there is no option to invest millions right now in sales and marketing. There is an inverse relationship between the differentiated value a product brings to the market and the amount of sales and marketing intensity it requires to sell it. The focus must be to leverage that differentiated value to acquire customers.

Richard Yau at BrightCellars has some interesting solutions to these issues, and they are all centered smartly around building a brand. In terms of building a brand of wine eventually, instead of reselling wine produced by others. He has two problems to solve for, one, how will he acquire new customers and two, how can he retain them, (remember, BrightCellars is a subscription service). The model uses an algorithm to predict what wines a person will like based off of other taste preferences. What are the customer acquisition and retention strategies BrightCellars can utilize?

  1. Content marketing
  2. Exceptional customer service
  3. Turn the recommendation engine into a viral loop

#3 above could allow BrightCellars to take off. How many times can they get users to utilize the recommendation engine? It has a singular and simple design that can be quickly iterated over to optimize for conversions and social sharing. How many situations does a paying user encounter organically that they want to “apply wine” to? Can each iteration through the loop be used to gather more customers to the service? The keys are to design the service around the core viral loop, rather than bolt on a “viral” component, and I feel like BrightCellars is in good hands.

All in all, it was an enjoyable day and I wish the best of success to the gener8tor Madison 2015 class!