Chapter 3 - Electronic Records and Storage

There are two main categories of electronic records: born digital records and digitized records. Born digital records are those records created with a computer that require a computer to be readable by people, such as a Word document, PDF file, an online post, or even emails sent. A digitized record is one that is born analog (paper) and has been converted into a machine-readable format using a scanner or camera, such as a scanned version of a paper record. For the purposes of this handbook electronic records refers to both types. The organization of electronic records such as email and word processing documents may resemble analog records, but more sophisticated electronic records such as geospatial records and databases exist which do not bear resemblance to analog records and will need special consideration to be managed properly.   

Electronic records contain metadata, which is the information that describes the content of a file and allows users to search and locate records using keywords which also needs to be accessible. Management of electronic records requires considerations of issues such as metadata management, storage optimization, electronic discovery requirements, and privacy attributes beyond the normal appraisal, use, storage, and disposition associated with the management of physical or analog records but proper management of electronic records necessitates. Metadata plays a significant role in electronic records management. Often defined as “data about data,” metadata’s function is essential for finding responsive records to electronic discovery searches and determining record authenticity. Specifically, metadata describes the content of a file and allows users to locate and evaluate data. Metadata is most useful if a structured format is in place using a controlled vocabulary.

Metadata Categories and Functions

There are several types of metadata to keep in mind. They will all prove useful in maintaining the content, context, and structure of your records and in keeping them useful and available for the long-term. Given the variety of digital records and the fast pace of changing technology agencies must consider options for ensuring access to records that an agency may want to retain for future use. Here are some functions and definitions to consider:

• Conversion- Converting a file such as a word document into a platform neutral format greatly increases the chances of having the file available for future use. One option may be to convert files retained for future use into PDF/A. Given the availability of programs such as Adobe Acrobat which easily convert files into a more desirable PDF/A format this approach is feasible. A records manager may want to inform or educate staff about converting files that may be needed by the agency for future use.

• Migration- Migration refers to moving a record or file from one platform, storage medium, or another physical format to another. For example, an agency will start to digitize hard copy records and start the process of transferring them to a cloud-based storage application. An agency may have active records that are stored on optical storage devices, such as CD, DVD and/or USB memory sticks. Migrating those records to more stable storage such as a storage server or optical disc is imperative to ensure availability for future use.

The needs of an agency and the electronic records that are identified as worth retaining will dictate strategies for long term records retention. Given the complexities of electronic records, it is best to consult with the agency’s Information Technology department on a solution that promotes long term availability of active electronic records.

• Descriptive Metadata –Describes a resource for the purpose of indexing, discovery, and identification. Common descriptive metadata fields include creator, title, and subject.
• Administrative Metadata – Helps manage a resource by describing management information such as ownership and rights management.
• Structural Metadata – Used to display and navigate digital resources and describes relationships between multiple digital files, such as page order in a digitized book.
• Technical Metadata – Describes the features of the digital file, such as resolution, pixel dimension, and creating hardware. The information is critical for migration and long-term sustainability of the digital resource.
• Preservation Metadata – Contains the information needed to preserve a digital object and protect the object from harm, deterioration, or destruction. Preservation metadata may encompass the mentioned forms of metadata.

One of the most pressing reasons an agency will want to create descriptive metadata is to ensure the discovery of information. If an agency receives a request by the public for information concerning the state of business, high quality descriptive metadata will make the search and retrieval of an electronic resource relating to the request much more manageable. An electronic resource with high quality metadata allows the user to identify resources, distinguish relationships with other objects, bring similar resources together, and determine location information.

Find more information on the Council of State Archivists (CoSA) website: CoSA: Metadata Standards.

Electronic Mail (E-mail) Records

The definition of a public record applies to email messages, and you want to make sure that your agency is accounting for all of the email records being created, even those that are kept for a very brief amount of time.

Email is not considered a record series or category on its own. It is simply a format. Emails should be organized instead based on the information they contain, the purpose they serve, and the relevant line item/records series to which they belong. If there is no entry that resembles or matches the subject matter of the message, the “record” should be added to the appropriate retention schedule as a separate series of records. Transitory e-mail consists of electronic messages that are created primarily for the communication of informal information as opposed to the perpetuation or formalization of knowledge. Destroy transitory email when it has served its purpose. Given the frequent use of email, it should be evaluated on a regular basis to ensure messages are being kept per the retention schedule. In simplest, terms you want to make sure that you are organizing your email by content on your retention schedule, not by format.  Individuals should configure their email filing to ensure accessibility to email records. Email systems should be configured so that email messages can be indexed in an organized and consistent pattern reflecting the ways in which records are used and referenced. 

Email Policies

An agency’s email policy should be developed to enhance management of record emails. An effective policy includes direction on topics such as email filing methods, email subject lines, and storage and retention of email, thereby increasing the accessibility of records. Policies should include whether the sender or the receiver should save email records, how to determine if an email is a record, and how to segregate record email into the appropriate series and record storage. Non-record and duplicate emails should be deleted from mailboxes regularly. If an agency receives a request for an email record for a litigation issue, for example, a well-planned email policy can help ensure that the record is discoverable during its retention period or show that its deletion was properly carried out according to retention policy.

Subject lines are helpful for both the recipient and sender in identifying and filing messages. They are also crucial for efficient email records discovery. Subject lines should be unambiguous and as descriptive as possible so that records are more accessible and searchable.

Poor or Confusing Subject Lines:	Better, Descriptive Subject Lines:
“Helpful Info”	“Contact Info”
“Report”	“Quarterly Financial Report”
“News”	“January 2001 Board Minutes”
“Minutes”	“Revised Administrative Procedures”
“Important”	“New Agency Head Appointed”
“Contract Status”	“PO 12345 Delivery Status”

Social Media Records 

Social media is a broad term that incorporates various web-based technologies such as blogging, video sharing, wikis, social networks, and photo libraries. Most state agencies operate one or more social media accounts which has added another dimension to records management. Tracking social media may be harder due to comments, metadata, or other information. 

Because these websites can be shut down at any moment in time, creating a plan on how to export records from a social media site to a record keeping system is important and should be created in collaboration with an agency’s IT department. If the public is able to interact with the agency via the social media platform, there needs to be a mechanism to track and manage all comments and questions in accordance with the retention schedule. We suggest keeping a limited profile where comments are disabled if applying retention to those records is difficult.

Social media create another form of electronic records. Since social media is a new way to communicate with members of the public, it can be difficult for agencies to determine how to account for what they are posting, identify what posts might be more important than others, and how to accurately account for something like a post or comment on any given platform, on their retention schedule. Some social media posts might be more valuable than others, but all of these posts constitute a public record and should be accounted for based on their content on your unit or agency’s retention schedule (RRS).

Unless the content created in social media denotes a new record series, social media records will most likely fit in the characterization of an existing record series, such as “Press Releases.

File Naming Conventions

A file name is the key identifier of a digital object and provides metadata for the record. Consistent and descriptive file names will provide a more organized and easily understood collection of records. How a file is named will have a large impact on finding the files at a later date and understanding their contents. The following information might be considered when creating a file naming policy although ultimately file names should reflect the purpose and need of an agency: Project name, Name of intended, Publication date, Date or date range of Audience, Release date creation, Description of content, Record series, Version number, Department, and Name of creator.

When creating a file naming policy, the following should also be considered:

• Create unique file names.
• File names should be easy to understand and not overly complex.
• Do not use spaces but rather (_) or (-) to represent a space.
• Avoid using special characters such as $ # @ & ^ % * ! and use only alpha-numeric characters.
• Limit file names to 25 characters or less.
• Use the three-character file extension with a period (e.g. .tif not .tiff) at the end of the name.
• Do not rely on the system to differentiate between upper and lower case and be consistent in what is used.
• If digits are included in the file name, include the appropriate number of leading zeros and be generous so your project can be scalable.
• It is helpful to include metadata in the file name butconsider using shortened versions of 1) a standardized date, 2) version number (only if this can/will vary), 3) creator’s name, and 4) description/type of document/subject in the file name and in a logical order.
• Document whatever naming convention chosen. Include the naming convention document any time records are transferred elsewhere.

Personal Digital Devices

Given the prevalence of portable digital devices, it is no surprise that many employees are using their personal devices to perform state work. Use of personal devices to conduct state business raises many issues and concerns for records management practices, and is not the easy, cost-effective solution that some agencies may consider it to be.

Data Management Concerns

A policy allowing the use of personal devices makes it difficult-to-impossible to ensure that proper information practices are being followed by the operator of the device. If an agency finds itself in a lawsuit, an auditor will want to know what steps the agency took to ensure that the data in question was adequately protected on an employee’s personal device.

Security Concerns

There is always concern about security when it comes to personal devices.

• Lost or Stolen Devices – One security issue that arises with the use of personal devices is the potential for employees to lose their personal devices containing unsecured data. Depending on the scope of work of the employee or the agency involved, sensitive information may land in the wrong hands.
• Malware – There is always the potential for an application downloaded to a portable device to contain malicious software that may compromise the device in use and any information contained on it.
• Disgruntled employees – Unfortunately, not all employees leave an agency on good terms and, if disgruntled employees have sensitive information on their personal devices, they may choose to publish or disseminate information not meant for the public.

Privacy Concerns

One concern an employee should have with a using personal devices is the potential for a breach in their privacy. If information on a personal device is needed for legal reasons, a search of the device will not only yield the state records but also any personal records such as email that may be on the device. All the information on the device would have to be preserved for discovery purposes and a “wipe” or complete erasure of data would not be possible because of the legal obligations involved with state records. An agency promoting the user of personal devices is potentially opening a Pandora’s Box of legal and privacy issues. If a portable device or laptop computer is needed for an employee to complete essential job duties, best practice would be to issue a state-owned device that has the proper IT support to accommodate security and discovery issues.

Digitization Projects

For more information about managing electronic records and digitization projects, please attend CalRIM’s self-paced course, RMA 106: Electronic Records and Digitization Projects, available for free on CalLearns.

Considerations Before Digitizing

The decision to begin a digitization project (or a project that entails the scanning of paper records into electronic formats) should be made by the team—the agency Records Management Coordinator (RMC), records management staff, division management, Information Technology (IT) staff, and records creators should all be involved in this decision. Once a decision has been made, the RMC must ensure they have notified the State Archives’ CalRIM team per Government Code section 12274(f).

Digitizing records can enhance accessibility, workflow, and productivity, but digital imaging projects can also be complex, time-consuming, and cost prohibitive. Besides high initial costs, digital images require proper management, including continuous maintenance to ensure the records are stored in a way that is trustworthy, complete, and durable for as long as they remain on an agency’s records retention schedule and for the possible transfer of records to the State Archives.

When you’re planning to digitize, consider the records’ use and access, whether you have appropriate storage for these new electronic records that will be created, and how the quality of these scans will be addressed to meet the needs of the agency (i.e., will these be taking the place of the paper copy, or will these be copies made to provide as deliverables?) ... Other concerns include record volume, preservation needs, legal restrictions, present and future storage, and appropriate storage formats.

Before beginning any scanning or digitization project, the RMC of your agency must be informed. They will assist in ensuring that the right stakeholders are in the loop, including internal agency management and the State Archives’ CalRIM team. In addition, the RMC can get input from the records users, records creators, IT specialists, records management staff, and any other staff that might provide valuable input. The following are some of the questions that should be addressed:

• What are the goals for the digitization project?
• What is the desired end result?
  • A document management system>
  • Document preservation?
    • Online search capabilities to facilitate better access to records?
    • What materials will be digitized?
      • Textual documents, photographs, or maps?
• How much material?
• What resources are readily available?
  • Will the records be scanned in-house? Do you need scanners? Software? Expertise?
• What file formats are most suitable for your records?
• What image quality is required?
  • Black and white? High resolution? What condition are the documents in?
• Is preparation required (e.g., removing staples and paperclips)?
  • What different type(s) of digital storage media will be used?
• What metadata is necessary for each file?
  • Is the metadata readily available or is time needed to gather information?
• How will access be provided to the digital records?
  • Intranet or internet? Offline? DVD, CD, Hard drive?
• How long will the digital files be retained?
• If the retention schedule dictates a prolonged retention period, what strategies for long-term document preservation must be employed?
• Will hard-copy documents be kept? What is considered the original?

Required Notification to CalRIM

Per Government Code section 12274(f), agencies must notify the Secretary of State (CalRIM@sos.ca.gov) when records are stored with a third-party vendor or digitized. Submit the online Record Digitization Notification form to meet the notification requirements. Agencies may also need to update their records retention schedule if the format of records is changing from what is currently listed for the affected record series.

Note: For instructions on updating the retention schedule, please see Chapter 6 of this handbook.

Digitization Standards

• Specifications for Scan Quality -- DPI and PPI

•  
• Per the NARA Digitization Guidelines:

“The spatial resolution and the image dimensions determine the total number of pixels in the image; an 8”x10” photograph scanned at 100 ppi produces an image that has 800 pixels by 1000 pixels or a total of 800,000 pixels. The numbers of rows and columns of pixels, or the height and width of the image in pixels as described in the previous sentence, is known as the pixel array. When specifying a desired file size, it is always necessary to provide both the resolution and the image dimensions; ex. 300 ppi at 8”x10” or even 300 ppi at original size. The image file size, in terms of data storage, is proportional to the spatial resolution (the higher the resolution, the larger the file size for a set document size) and to the size of the document being scanned (the larger the document, the larger the file size for a set spatial resolution). Increasing resolution increases the total number of pixels result[s] in a larger image file. Scanning larger documents produces more pixels resulting in larger image files. Higher spatial resolution provides more pixels, and generally will render more fine detail of the original in the digital image, but not always. The actual rendition of fine detail is more dependent on the spatial frequency response of the scanner or digital camera…, the image processing applied, and the characteristics of the item being scanned. Also, depending on the intended usage of the production master files; there may be a practical limit to how much fine detail is actually needed.”

• Ensure A Faithful Digitized Reproduction of the Records

•  
• Identify before scanning: are you planning to have the electronic scans take the place of the paper records, or are you scanning records for reproduction purposes?

The National Archives has some helpful information about various formats and format specifications that electronic records could be kept in. Agencies want to ensure that they are aware of the basic standards required of retaining records in specific formats so that those records are not damaged or lost.

• The National Archives has some helpful information about various formats and format specifications that electronic records could be kept in. Agencies want to ensure that they are aware of the basic standards required of retaining records in specific formats so that those records are not damaged or lost.

We have listed the National Archives’ guidance for scanned, paper text documents, which is taken from their site at https://www.archives.gov/records-mgmt/policy/transfer-guidance-tables.html#scannedtext.

Keep in mind that the initials DPI stands for “Dots Per Inch”, which is one way of specifying how much detail is preserved in a scanned image. PPI stands for “Pixels Per Inch” and is the digital equivalent to DPI—it’s basically the measurement of how many pixels per inch there is on a page, which will change the overall quality and clarity of the scan being created.

Agencies must digitize to standards appropriate for the accurate preservation of the information on the printed page. When converting analog or film based material (microfilm, microfiche, slides, etc.), agencies must digitize to standards appropriate for the accurate preservation of the original image. Examples of appropriate methods and formats are available on NARA’s Digitization Services Products and Services page, which is available at https://www.archives.gov/preservation/products/definitions/textual-microfilm-def.html.

Electronic Records Replacing Paper Originals

 

Is the goal of your digitization project to reduce the amount of paper in your office? Are you wanting to create the electronic version of the paper record, have the electronic record be the official record, and then destroy the paper version? If so, it is very important that quality checks of these scans are occurring, and that these scans are being done with the correct quality settings. Consider scanning at these settings if the electronic scans will be taking the place of the paper originals:

The National Archives Federal Records Management page is incredibly helpful in providing digitization information based on your records’ format.

• A well thought out scanning project must provide direction and standardized procedures for entering metadata for scanned images to ensure uniformity with all objects involved in the digitization project. Metadata, often referred to as “data about data,” describes the characteristics of the object and provides meaning, context, and organization. Metadata is an essential component of a digital imaging project. Complete metadata will allow searches by subject heading and keyword. The value of metadata is especially evident when documents are requested for litigation purposes: the ability to locate documents quickly and efficiently can save an agency time and money.
•  
• Once digitization is complete, a thorough review of the resulting files should then occur. Once all files are deemed acceptable by all parties, it may be time to send the hard copy files to the State Archives if they are records that were flagged for transfer. State Archives staff members are also available for consultation.
•  

• Quality Control

•  
• It is very important to ensure not only that the scans are being made, but that they are quality scans, meaning that they are legible, clear, and saved with the right DPI (dots per inch) settings. If it has not already been established, RMCs should work with management and RMACs during a digitization project to ensure there is someone identified to do quality checks on scans and ensure that they meet minimum quality requirements.

Management and Preservation of Digitized Documents

How long an agency will maintain custody of digitized documents depends on both operational needs and the required retention period of the record. Managing and preserving electronic records requires a systematic, sustainable, and ongoing plan. Maintenance of electronic records can be just as costly and time consuming as creating the records in the first place. Data recovery for records that have not been maintained properly is also very expensive.

Periodic checks to see that files have not been damaged or altered, migration to new formats, and transfers may all be necessary depending on the situation. Below are two additional resources that may be helpful in summarizing best practices:

1. State Agency Electronic Records Tips (PDF)
2. ARMA: Generally Accepted Recordkeeping Principles

Trustworthy Systems and Internal Staff Access

All electronic records, whether they are born digital or digitized, should be properly stored in a trustworthy system. Establishing a trustworthy system is not simply purchasing a program and downloading it to your computer, but rather evaluating agencies’ storage processes and systems to meet minimum requirements.

Trustworthy- This refers to the information that is retained in your records. Is it reliable, authentic, and unaltered? Will your records hold up in a court of law, if necessary? A collaborative effort with your IT department to identify and implement a strategy to ensure authenticity and trustworthiness of your records may prove valuable if the records in question go before a judge. A trustworthy system is a set of policies, procedures, and technologies used to ensure that electronically stored information can be trusted as a true and accurate copy of the original information put into the system. In other words, a trustworthy system certifies that electronically stored information (ESI) is an authentic copy of the original document or information. These records must be able to stand as the true and authentic copy of the record in the case of a Public Records Act Request (PRA), audit, or lawsuit. 

A trustworthy system must include an avenue for maintaining at least two separate copies of an electronic resource. A combination of proper hardware and media storage techniques are necessary to prevent any unauthorized additions, modifications, or deletions to a document. A trustworthy system must also stand up to the rigors of an independent audit process that ensures that no plausible scenario for altering documents is feasible. Lastly, a trustworthy system requires that at least one copy of a stored electronic document or record does not permit any unauthorized alterations or deletions and is stored and preserved in a separate and safe location.

Establishing a trustworthy system is a team process, and will take group effort between agency staff, agency management, and the agency IT department to establish. There are various laws and resources that exist to assist you in establishing your agency’s trustworthy system:

Here are the various requirements for Trustworthy Systems if one needs to be established (if so, this will require collaboration with your RMC, IT team, and with your management):

• Government Code section 12168.7 sets the requirements for a trustworthy system and makes those requirements mandatory for state agencies.
• The Secretary of State has also published regulations under 2 CCR 22620.1.  
• AIIM ARP1-2009 – Analysis, Selection, and Implementation of Electronic Document Management Systems (EDMS)
• The ISO documents referenced in the Government Code can be requested from CalRIM by email at CalRIM@sos.ca.gov.

When considering internal staff access to records, this requires a plan for once the records are digitized.

• How will access be provided to internal users?
• Will any specific terms or conditions be configured into the system to allow or restrict access?
• Is there a filing plan in place for staff to follow to ensure that records are being organized based on their retention schedule, and that records nearing the end of their retention be easily identified in order to determine if they will be transferred to the State Archives or destroyed?
• How will non-public content be secured? If records are to be presented in court, they generally must be certified. Electronic records can only be certified if they reside in a trustworthy system as specified in state regulations (2 CCR 22620.1-22620.8). The infrastructure required for such a system is costly and requires long-term planning and budgeting.

Agencies need to first have a trustworthy system in place to store their electronic records safely. Your agency’s Information Technology (IT) team or division can help build a trustworthy system if one does not exist for your agency already. If you are the RMC of your agency, and there is not a trustworthy system in place (or the current one could use some optimization), work with your manager and your IT team so you can be part of the conversation. You bring the records management perspective to that process!

You can find more useful tips and recommendations in the Trustworthy Repositories Audit & Certification: Criteria and Checklist guide from the Center for Research Libraries and the Online Computer Library Center.

Electronic Data Management System (EDMS)

An Electronic Document Management System (EDMS) is a software package designed to manage electronic information and records within an organization’s workflow. Utilizing various technologies, an EDMS allows a user to manage the creation, storage, and control of records. An EDMS can automate processes and increase efficiency. Before adopting an EDMS, it is necessary to determine how it will fit in with your agency’s records management program. It is not a replacement for sound records management practices.

EDMS Functions

Many different EDMS systems exist. Each provides various functions tailored to specific and unique needs, but all EDMS systems should include the following basic functions:

• Security Control – This feature is crucial to control access to information. A system should have a mechanism to safeguard documents that are exempt from disclosure and allow access to those records which should be made publicly available.
• Addition, Designation, and Version Control – The EDMS should allow users to add documents to the system and designate a document as an original government record. It should also automatically assign the correct version designation.
• Metadata Capture and Use – The EDMS should allow the user to capture and use the appropriate metadata according to an agency’s needs.
• Optional Functions
• Records Management – Not every EDMS is equipped with records management capabilities. Systems with a records management component are sometimes referred to as an Electronic Document and Records Management Systems (EDRMS).
• Storage – An EDMS may provide storage within the EDMS or the ability to work with an adjunct storage system.
• Free-Text Search – An EDMS may allow users to search every word in an entire document while other systems only provide metadata searching capabilities.
• Automatic Conversion – This function provides the user with automatic conversion of a document from one file format to another (e.g., from a Word document to a PDF) after the file has been designated as a record.

Benefits of an EDMS

Most state agencies create, collect, process, distribute, store, manage, retrieve, maintain, and dispose of enormous amounts of electronic information. An EDMS may improve efficiency and effectiveness of an agency by:

• Improving Access to Records and Information – An authorized user can search documents within an EDMS and workflow can automatically notify a user when needed information has arrived or been processed.
• Improving Customer Service – Retained information can be immediately accessed by a user and easily transmitted to a customer whether it is a member of the public or a representative from another state agency.
• Minimizing Duplication – A single copy of a document can be made available to all authorized users. Knowing that only one copy exists is especially useful when disposing of records. Business Process Automation – Certain processes that were once done manually may be performed automatically by an EDMS.
• Regulatory Compliance Improvement – Compliance with records retention schedules can be automated and improved with incoming documents being automatically classified and stored by the system.

Selecting an EDMS

When preparing to select, implement, and manage an EDMS for an agency, appropriate stakeholders should be identified and brought to the table for discussion. Establishing and assessing the needs of an agency should be the first order of business. Each agency will have a unique set of needs depending on legal obligations and records management strategies, but trustworthiness, completeness, accessibility, legal admissibility, and durability are all critical (these concepts are further explained in the “Key Concepts” section of the guidebook.) The type of records an agency creates now, and in the future, should be considered when examining options for an EDMS along with the following questions:

• What are the current and future needs of all involved stakeholders?
• How will the EDMS system be used? Solely for workflow or also for records management?
• What types of records will be captured and managed using the EDMS?
• Will existing records be migrated into the system?
• What type of metadata should be used and who will manage it?
• What records need to be shared and stored? How will records be stored and organized to better facilitate retrieval and access?
• Records should be properly filed before system implementation.
• How will records be disposed of from the EDMS?
• Does the system facilitate the ability to easily transfer, convert, or migrate records?
• What features are most essential or desired by the agency? Are other useful but non-essential features desired?
• Will the new EDMS integrate with existing systems? (i.e., email systems, databases)
• Implementation and Deployment of an EDMS

Once an appropriate EDMS is selected, internal stakeholders and the EDMS vendor must develop a comprehensive implementation and deployment plan. A plan will outline how and when the system will be installed, and tested, as well as provide a background of the system. The workflow should be documented and tested. Consider allowing eventual users of the EDMS to participate in the testing of the system while soliciting their feedback. A training program and procedures should also be developed before the system is fully deployed to provide users with the necessary tools to use the system to its maximum potential. Ongoing management of the system will also be necessary.