Certica offers Navigate Item Bank™, a collection of assessment items and prebuilt assessments used by schools to enhance formative assessment programs. Creating and managing this collection is a hefty undertaking that includes planning new content, maintaining and updating existing content and learning standard alignments, and regularly deploying content and associated materials.
After Certica acquired ItemLogic™, the item bank team capitalized on the opportunity to migrate content creation, management, and deployment operations to ItemLogic. In this post, Carrie Frech, Director of Content Products, shares how her team and the product have benefitted from using ItemLogic.
In the past, how have you managed the people and resources used to create Navigate?
Our content specialists manage an integrated team including item writers, item reviewers and copyeditors, artists, standard aligners and metadata specialists that all contribute to the development process. This large team is distributed across several states and time zones, so it can be challenging to collaborate with and manage such a distributed workforce.
Previously, items were created in a home-grown system, but several other applications were also needed to plan, design and review items including Microsoft Word, Excel and Access, Trello and Google Docs.
How do you deliver the item bank to partners?
Our release deliveries include:
- Items in QTI 2.1 format
- Metadata tags (Excel spreadsheets)
- Alignment files (Excel spreadsheets)
- Coverage reports (PDF documents)
What are some challenges you have faced in the past? How has ItemLogic helped you solve them?
Content management and development occurred outside of our home-grown system. Multiple apps were used, and processes were developed just to prepare content to be entered in the system. Because learning standard alignment information could not be stored with the items, planning required the use of Access databases and Excel spreadsheets to determine which standards to target for item creation. Items were designed in Word documents and review and approval was done using Trello. Once that part of the process was completed, the items would then be entered in the system. Specific team members were assigned the task of entering the approved items in the system to cut down on the number of people that needed to be trained to use it. All standard alignment and metadata reports were created using Excel and Access. This process was cumbersome, and it was challenging to keep the content organized. Even simple tasks such as looking up the standard alignment for an item required the use of multiple databases and applications.
ItemLogic allows us to use one single platform for the entire item development process. Dashboards and analytics provided by ItemLogic help us understand opportunities for product development, and blueprints let us plan projects and assign tasks. Our contributors create items and passages directly in the application—tagging them in real-time with standards and metadata before assigning to a team of reviewers and copy editors. Once approved, items, metadata, and standards information are prepared for delivery.
Item search and standard alignment were cumbersome and limited. Because item identification numbers and standard alignment information were maintained in spreadsheets, anytime we needed to search for an item by its alignment, we had to cross-reference this information in a spreadsheet before looking up the item in the system. Additionally, we could only align one standard from a given standard authority to an item. While most of our items are written in a very targeted manner to a specific standard, some item types such as writing prompts, align to multiple standards.
ItemLogic autogenerates identification numbers when an item is created and because standards and metadata are tagged to the item, we can use multifaceted search to locate items quickly and easily. Furthermore, ItemLogic doesn’t limit the number of learning standards aligned to an item and lets us leverage the Academic Benchmarks repository of standards metadata.
The need for a reliable way to preview items. Our system provided limited capability for previewing an item as it would appear in an assessment platform or learning management system. This made it difficult to review items and anticipate how students would be interacting with them on a test.
ItemLogic provides us with an interactive preview of items, letting our authors and reviewers experience the item as a student would, while also displaying, standard alignments and associated metadata in the same view. Not only does this increase our confidence in the item, but it also simplifies the review process.
Content delivery and distribution were cumbersome. Preparing content for release required outputs from multiple sources including databases, spreadsheets and the item QTI files from the system. It took time to gather and QA the files and then package them up for partners.
ItemLogic simplifies this process—items, standard alignments, and metadata are output together. We also have expanded our delivery capabilities to include API rendering and scoring – which allows us to be more flexible and provide more frequent deliveries.
What does your development team think of ItemLogic?
The team now develops items directly in ItemLogic—and many contributors have commented about how easy and intuitive it is to create items and manage resources such as passages, images and video. The learning curve was short, and they were able to quickly create new content.
What are the biggest benefits you experienced when adopting ItemLogic to develop the item bank?
The biggest benefits of our transition to ItemLogic include:
- Intuitive item authoring that provides a scalable process and lets us create more content in less time.
- A collaborative review process that lets multiple members of our team review and comment on items in real time.
- Integrated content and standards, which impacts many of our processes from planning to deployment.
- Comprehensive content management capabilities including audit trails, multifaced search and the ability to apply changes to multiple items at one time make it easy to maintain the item bank.
We had to use many different applications and design processes to supplement the features of our internal system–ItemLogic lets us eliminate the extra applications, create more content in less time and quickly deploy content updates to the classroom where it can have the most impact.