Meeting Minutes: October 2013

Society of American Archivists

DAS Subcommittee Meeting

March 1-2, 2013

Chicago, Illinois

Minutes

Friday, March 1 – 2:00 pm to 8:00 pm

• Welcome/Introductions – De Sutter

Present:  Liz Bishoff, Veronica Martzahl, Mahnaz Ghazanvi, Glen McAnninch, Lori Lindberg (present via speakerphone), Subcommittee Members; David Kay, Naomi Nelson, Education Committee Members; Geof Huth, SAA Council Liaison; Solveig De Sutter, Director of Education; Hannah Gdalman, Education Intern. Sandy Dolan, Psychometrician Absent: Jackie Esposito.

• Sandy Dolan
• Standard Setting – Presentation will follow along with handout
  • Setting a minimum passing score will be the final result.
  • Confidentiality
  • Typically certification exams don’t have a specific course requirement (taker is left to acquire knowledge on their own), a certificate does.
  • Page 4-Questions (Mahnaz): Electronic records and born digital records. Are these perhaps a subset of a broader umbrella term? “Anyone who has a responsibility for managing electronic collections.”
    • Consensus is that this should be put off to another time. Language used on the SAA website warrants changes to be more inclusive. What is it that people need to know to pursue this program?
    • Electronic records are part of the picture, but what about manuscripts?
    • We need to be more all-inclusive: digital collections instead of digital records.
    • Consensus for today’s work: we will be looking at the exam using the more general term of “digital collections” to avoid problems with marketing, etc.
    • For how long the exam takers will have, Sandy Dolan suggests at least 3 hours. The pilot-question beta version perhaps more time like 5 hours to account for the additional questions.
    • Liz Bishoff questions whether we’re setting our bar too low with the “High-school diploma” caveat. Geof and others argue that it doesn’t really matter aside from that the reading-level of the exam is set at the 12^th grade reading level. Has nothing to do with the content of the exam.
      • Consensus is to remove the education requirement altogether since it doesn’t really matter and perhaps sends the wrong message. 
    • Tools/proficiencies/skills (page 5): which one?
      • Consensus to call these “Key concepts”
    • Page 6: Crux of the discussion today. What would the barely passing examinee know?
      • What other characteristics of the examinees may come into play here?
        • Students or young professionals. Teachers of history/social sciences, lawyers, humanities professors—second career archivists.
        • Expectation that these people have at least had minimal experience working/volunteering in the field? Or perhaps that they are generally lacking any hands-on experience (students, career-changers, people already in the field looking to re-tool, etc)
        • Consensus is that this exam is aimed to measure minimal competency and NOT a level of excellence.
        • Plus, a thorough understanding of archival practices.
        • Sandy: So the basic assumption is that the people taking the exam have not only taken these 9 courses but also have at least some exposure to/experience within the field.
    • Page 7:
      • Task 1, Item 1: Clear failures may be able to name a few steps in the process, but would leave out steps. Lacks ability to conduct background/environmental research. Works alone. Would know a model to use, would be unable to identify formats and standards to use.  Unable to relate it to any kind of broader plan.
      • Task 1, Item 2: Does risk assessment as well as needs assessment.
      • Task 1, Item 3: Know it in theory but maybe not in practical application. Be able to produce minimum recommendations, a basic report. Lacks skill in facilitating focus groups, team meetings. Would not do the analytical work. They can create lists, do background research. They would lack in engaging stakeholders. Interpretation and delivery would be lacking. They’d have in line some sort of model; the basic steps along with at least some of the appropriate tools. Able to bring people together, document current situation and compare solutions and evaluate them but they would still be missing a certain amount of information.
    • Page 8 - Task 2, Item 3: Barely passers know what they need to do but they don’t necessarily know how to go about doing it in the best way. They probably don’t know the whole range of the process.
    • Page 9 - Task 3, Item 3: Recognize some models for repositories services and name them.
    • Page 11 – Trial Items
      • 85, 85, 80, 50, 50, 95, 40, 75
        • Slight contention on if this is the percentage of “barely passing” examinees who SHOULD or WOULD get this question correct. This isn’t an exact science.
        • This should perhaps be higher since the question involves “is best” in the stem.
      • 85, 80, 90, 60, 85, 70, 90
        • Discussion on how “core” this is. People who are focused on electronic records may get this wrong because of the nature of the question.
      • 75, 80, 90, 75, 70, 80, 60, 80
        • Have decided to edit this question: “Within the management of a digital archive, a key role of the archivists is to:” Change to “Within the development of a digital archive, a key role of the archivists is to” answer b is “participate in setting functional requirements.” (Tact and Strategic line 35)
      • 75, 80, 80, 50, 70, 65, 40, 50
        • Contention between A and C for the correct answer. In the spreadsheet it’s marked as C. Liz Bishoff wrote this one. This is tough item so ratings should be lowered accordingly.
          • Sandy emphasized, and re-emphasized, the importance of considering the imagined examinee as a “barely passing” candidate. Therefore it would be expected to have the whole range of scores on the exam, from 0-100%, depending on the difficulty level of the question at hand.
  • On to the Real Deal, rating of the 115 questions, one by one:

• Taking the questions in chunks now. 21-31, then 31-50, etc
  • **Other changes needed to items:
    • Item:
    • 45: Cut out “Metadata records include general
    • 41: Add “is” or other verb on end of question.
    • 42: Answer a: Needs revision A “to restore data following loss” and D “assure long access”
    • 34: Answer is actually a maybe??
    • 45: Remove first sentence in the stem.
    • 47: change “collections” to “materials” in title and Option A is “ Section 108a of copyright law”
    • 49: change choice D from SAA to IMLS
  • The meeting was adjourned at 8:00pm in agreement that tomorrow Solveig will take on Sandy’s moderator role to work through the remaining questions. Paperwork will then go to Sandy. She will process everything into a formal report with descriptions of passers, barely passers, etc.  She will enter scores into a spreadsheet and take averages of the FINAL scores, which will then become the minimum passing score of the exam.  The report will include all the ratings.  Once exam is given, the data compiled now will be important for referential value.
  • First batch of examinees will also be given a profiling survey after the exam. Sandy asks that DAS Subcommittee members submit any recommendations for questions (both multiple choice and open-ended) for this survey.
  • Agenda for tomorrow is to finish rating these questions and then look at the reviews made on the questions. If time afterwards, there will be a meeting of the DAS Shepherds. 

Saturday, March 2 – 8:00 am to 4:30 pm

• Welcome– De Sutter (8:19am)

Present:  Liz Bishoff, Mahnaz Ghaznavi, Veronica Martzahl, Glen McAninch, Subcommittee Members; David Kay, Naomi Nelson, Education Committee Members; Solveig De Sutter, Director of Education; Hannah Gdalman, Education Intern. Absent: Lori Lindberg, Jackie Esposito

• Veronica continues discussion of question 34: After doing some research she believes the answer is in fact A. Consensus is that the question needs to be re-worded or pulled entirely. “What type of Standard is DACS…”
  • Decided that this should be a pilot since it’s perhaps not the best question.
  • We have to decide which “pilot” we will then swap out for this one.
  • Scoring of items 50-75:
    • Item 57: Metadata is one word.
    • Item 66: Perhaps a typo? “Structured” or “Structural”
    • Item 70: Sic. (option C)
    • 9:58 am: Scoring of items 76-100
      • Item 76: Does this question apply since it’s not directly related to Digital?  Consensus is that yes, it does more loosely apply to Digital and is therefore important.
      • Item 77: Geof proposes change to question: “When creating a trustworthy environment, you need to”. Liz wrote this question based on a slide from her class. “Trustworthy environment” is a term pulled directly from the slide.
      • Item 79: Flawed. Option D “incorporating the digital program in the larger environment”?  Liz wants to look back on her slides to see about the wording here.  No immediate changes.
      • Item 82: Change to option A “working with the archivist to establish content creation standards”
      • Item 84: Change to specify. “The duration of copyright in U.S. copyright law is”
      • Item 85: Remove quotes. Option D insert comma between “immediately” and “prior to”
      • Item 86: Option B “Off site” not “Off-site”
      • Contention between use of “organization” versus “institution” in the exam. Agreed that “institution” is the better word.
      • Item 89: Option C SHOULD be the correct answer.
      • Item 90: Mahnaz and Geof think it’s a flawed question. Remove one “about” from question. “Supporting a set of services” instead of “Building a new set of services”
      • Item 91: Too wordy. Remove “Archival Information Packets” from question. Just use AIPS.
      • Item 92: Change wording in C And D. B should be “demo ability to install a preservation system.” C “Find the program” and D “Hire staff”
      • Item 94: Capitalize “Data Object” in option B for consistency
      • Item 100: Change options to remove all “considerations”
      • Item 102: Change option A to “the first sale rights provision of US copyright law.”
      • Item 104: Option C fix spelling on “Original”
      • Item 105:  In question, “Checksums” is one word.
      • Item 106: “Which of the following is a best practice approach to em…”.  Add hyphens to answers.
      • Item 108: Remove “reversible” from question.
      • Item 112: Option A: packages, B: “A single location system” and D: “offline storage”
      • Item 113: Question: “When developing a new tool or website, a preferred strategy for creating…”
      • Item 114: Option A: lower purchase cost.
      • Reviewing the reviewers comments over lunch (Naomi and Geof leave):
        • Change over 61 to a  non-pilot to switch with 34 (which will be a pilot)
        • Talking about Shepherded courses:
          • Glen’s phone-based survey had a positive impact. Can replicate this?
          • Mahnaz has idea of writing an article about the DAS program. Mahnaz will write outline and send it to the group for feedback
          • Glen’s concerned that he scared some people away with his questions. (Talking about Accessioning and Ingest) Glen offered to take on a mentor role to the instructor.
          • Discussing challenges of shepherding:      
            • How to offer advice/suggestions to instructors without offending them/stepping on toes (as in Glen’s situation)
            • Instructor deadlines?  How to get instructors to respect guidelines, etc.
            • Mahnaz brings up point that there is no “best practices” guide
              • After reviewing our “best practices” guide, Liz thinks it should be an over-arching “How to make an SAA PowerPoint” guide.
              • Solveig coins new term, SME’s (pronunciation shmee), subject matter experts
              • Faculty session: First hour for information. SAA can explain where we’re at administratively, etc., etc. Try to give the “big picture.” Then section off 45 min to an hour for discussion, small groups, etc.
              • Will send these “Stream of consciousness” minutes to Lori and Jackie to review, then will re-group.

The meeting was adjourned at 3:00pm.