Jump to content

Creating a Load File in Intella


jon.pearse

Recommended Posts

Hi all,

 

We have been asked several questions on the support portal in regards to how to configure load files. Unfortunately most of the requests come in after a load file has been created and issues have been identified with the load file during the data review stage. Some common issues are:

  1. The user is inexperienced and is struggling with the concept of load files and how to create them. A load file has been created and provided to the client with the wrong specifications.
     
  2. There has been little communication between the parties and there has been no agreement regarding the requirements/specification for the load file. A load file has subsequently been created and provided to the client with the wrong specification.
     
  3. The user has not sent their client 'test' load files to ensure that the specification and other aspects of the load files are correct. The client is now reviewing data and has identified issues with the load files.
     
  4. The user accepts a load file to import without having correctly done the diligence of defining the fields and settings required with the other side / your client. 

The issues above are worst case scenarios, and will potentially alert the other side / your client your lack of understanding on the process of ediscovery and load files. Fixing issues after the review process has started can be very complex and costly. For that reason we have created a video on creating a load file using Intella so that users can gain more knowledge before they start building load files. The video demonstrates getting your dataset ready and exporting the data to a Relativity load file.

 

Note: This video is for demonstration purposes and is only a guide to show you what may be involved when creating a load file. In the video we have created a simple load file and we have not demonstrated every setting available in Intella.
 

Creating a load file can be an intricate task. The requirements for fields and load file settings need to be agreed with the end user, and you will need to ensure that these settings are met. These setting can change between clients and projects which makes creating each load file more of a unique task. 

 

Load files require you to have knowledge of not only Intella but also the software you are creating the load file for. There are a number of (non-Intella related) items which you should be aware of. Some of these items are load file concepts, field naming, document numbering, date structure/formats, family structure, the ultimate use of the load file, encoding and jurisdiction.

 

If you have not created a load file before, have little experience in this area, or are not aware of the non-Intella related items mentioned above, we suggest that you seek assistance from someone who has experience in load files. They should be able to guide you through the many areas where things can potentially go wrong. 

 

https://www.youtube.com/watch?v=bxKSodX63_M&feature=youtu.be

 

For more information on load files and Intella. Please see section 27 "Load File Checklist" of the user manual. 

Many thanks to Jason Covey for assisting us with this video.

 

Jon

Link to comment
Share on other sites

  • 6 months later...

Hi all,

 

We have been asked several questions on the support portal in regards to which fields a user should use in a load file. We can't really tell you which specific fields to use for your load file as we are not involved with the investigation/litigation. However, what we will do in this post is provide you with more information in regards to which fields are available when creating a load file and how they can be used.

 

The fields you should use in a load file will be determined by your client or the firm who you are creating the load file for. Before creating a load file, you should have already discussed the requirements with the receiver and agreed on a specification for the load file (see the 'Creating a Load File' video in the previous post). We recommend using the check lists in the user manual (the Load File Checklist section) for discussing and agreeing on a load file specification with your client.

Note: These check lists should be used as guides. Some load file requirements may not be shown on these check lists. It is the load file creator's responsibility to ensure that all aspects of the agreed specification for the load file is met.

 

This post is more about the fields which are available and how they may be used. In Section 25.2.11 of the user manual you will see a number of load file options. What we are discussing here is the list of fields shown toward the end of this section in the user manual.

 

Intella Metadata Fields

The most commonly used field from this list is INTELLA_COLUMN. This allows the user to use any of the Intella metadata fields that are shown in the column chooser window for the Details table.

 

Although the Intella metadata fields are mostly self explanatory and documented in the user manual, there are some fields which require some additional explanation.

  • Any date field can be exported in three different formats:

Date Only

Time Only

Date and Time

 

01_date_select.jpg

 

Some load file specifications require that the Date and Time fields are separate. This can be achieved by creating two fields. For example, a sent email would have one Sent field with the 'Date only' setting and the another Sent field would have the 'Time only' setting. Note that it is important to set the date and time format correctly in the Load file options window prior to creating these fields.

 

  • The Intella metadata field for 'Page Count' can produce unexpected results. Note that this data comes from the document's metadata, so if the document doesn't have this property, Intella will produce '-1' and show this in the field. Also, this property only works for MS Office and PDF documents.

A better alternative is to use the NUMBER_OF_PAGES field as it is not dependant on the document's metadata. This property will also work with any type of document, as it is determined during image production. Note that with this setting, if additional metadata (comments, attachments etc.) is selected during the export process, the number of pages for the exported document may increase.

 

 

Non-Intella Fields

Let's have a look at some of the non-Intella metadata fields. We have provided more detail about these fields and what they do.

 

Document Numbering

  • RECORD_ID_START: This displays the name/number of the first page of the document. It is the main ID for the document and is commonly also referred to as BegBates or DocumentID or the Identifier or the Control number.
  • RECORD_ID_END: This is the name/number of the last page of the document. For example, if a 5 page document started with ABC0000001, the RECORD_ID_END value would be ABC0000005.
  • RECORD_ID_GROUP_BEGIN: This is the RECORD_ID_START value of the first page of the first document in the current "parent-child" group. It would be the first page of the most top level item in that group. This field may be required for email threading and other processes used in Relativity and is sometimes referred to as the Group ID.
  • RECORD_ID_GROUP_END: This is the  document ID of the last page of the last document in the current "parent-child" group.
  • DIRECT_PARENT: This is the RECORD_ID_START value of the document's direct parent. This is commonly referred to as the Parent ID for a document. Note that there is a similar field named RECORD_ID_PARENT. This shows the RECORD_ID_START value for the top level document which is different to the RECORD_ID_START value for the direct parent.
  • DIRECT_CHILDREN_IDS: This shows a list of RECORD_ID_START values for the document's direct children. Note that there is a similar field named ATTACH_ID_LIST. This show all of the RECORD_ID_START values for all child items of a particular item, not just the direct child items.

 

Note:

When showing Parent and Child IDs, you may be required to do this on a single level basis. For example, you may need to show the Parent ID of a specific item, one level up. And, you may also need to show the Child IDs for a specific item only one level down. In this case the   DIRECT_PARENT and DIRECT_CHILDREN_IDS fields should be used. An example of this is shown below where Document ID = RECORD_ID_START, Parent Document ID = DIRECT_PARENT and AttachIDs = DIRECT_CHILDREN_IDS.

 

02_table_view.jpg

 

  • BEG_ATTACH:  This shows the RECORD_ID_START value for the first page of the first attachment document in the current "parent-child" group. This will be empty if there are no attachments in the current group. This is used for emails only.
  • END_ATTACH: This shows the document ID of the last page of the last attachment document in the current "parent-child" group. This will be empty if there are no attachments in the current group. This is used for emails only. 

 

Native, Text and Image Locations

  • FILE_NATIVE: This field shows the relative path (from the load file DAT and OPT files) for the original native documents. Usually a folder named Native (which holds the native files) is placed in the folder where the load file DAT and OPT files are located (e.g. Natives\01PREFIX00000001.doc). This allows the Natives folder to be accessed for original documents when the load file is imported.
  • FILE_TEXT: Similar to FILE_NATIVE, this field shows the relative path (from the load file DAT and OPT files) for the text files that contain extracted and OCRed text.
  • FILE_IMAGE: Again, similar to FILE_NATIVE, this field shows the relative path (from the load file DAT and OPT files) for the rendered images or PDF files.

 

Note:

Although the paths for text files and image files can be different, some people request that these files are both exported into a single folder. For example, in case where this is required, the paths for text and image files may look similar to:

o   Images\01PREFIX00000001.txt

o   Images\01PREFIX00000001.tif

This is entirely up to the person requesting the load file. Intella provides flexibility in this area to accommodate any formation.

Link to comment
Share on other sites

  • 2 months later...

Creating an overlay in Intella

We have received a number of support tickets regarding creating load file overlays with Intella. An overlay is used to add additional data to a load file which has already been ingested into a review platform. For example, the case may be that a load file was created in Intella and provided to the end user. The end user ingests the load file into their review system (such as Relativity or Concordance) and they discover that some metadata fields (which they required) are not in the load file. The solution is to create an overlay so that the missing metadata fields can be overlaid on to the records that have been loaded into the review platform from the original load file.

Intella does not have an 'export overlay' option as such. That said, you can export a trimmed down version of a load file that can be used for the purpose of overlaying metadata onto existing records. The so called 'overlay' will contain the missing metadata fields for each record (based on the original load file that was provided to the end user) and will be overlaid onto the existing records by using a common reference or identifier such as the DocID.

The key point mentioned above is that the records are updated by using a common reference such as DocID. If an 'Export set' was created when the original load file was exported from Intella, the Export set can be used as the common reference and creating the overlay is quite simple. This post shows the steps involved. However, if an 'Export set' was not created during the load file export, then this can be many times harder to do. If that is the case, then really the only other unique identifier you could use (if it was included in the original load file) is Intella's ItemID.

In these situations the review platform may allow flexibility as to which field can be used as the reference when loading an overlay. If this is the case then the ItemID can be used without a great deal of effort. If the review platform only allows the DocID to be used for the reference, then the DocIDs will need to be added to the overlay manually. This could be done by matching the ItemIDs in the original load file with the ItemIDs in the overlay, and copy and paste the load file DocIDs for the matching records in the overlay. This will be a manual process and is subject to human error, so adequate quality control and checks are required. This post is not for this situation, but rather explains how to create an overlay when an Export set has been created.

Below are the steps to create an overlay. Note that this is an example only, and your overlay may require different fields and data. You should build your overlay to match the requirements.

 

1) You start by selecting the same items which are in the original load file. As creating a load file, right click on the items and select 'Export highlighted items'.

 

2) The 'File naming and numbering' settings could be anything. We will discard these setting so they are not important.

 

3) Under 'Load file options' we can turn everything off (unless you are adding date/time fields). Turn off natives, text and images as these would have been included in the original load file, so they are not needed here.

image.png

 

4) Under the 'Load file chooser' options create a field for the Export set that was used for the original load file export. Also add the metadata fields that you want in the overlay. In my example I have added the fields for the Export set, To, From, Subject, Primary date/time and File name. Your requirements will probably be different, but the important thing here is to make sure that the Export set is included as this will be required for matching the records.

image.png

 

5) The 'Redacted items' screen will be greyed out as we are not exporting natives, extracted text or images. Run the load file as normal.

 

6) The load file (or overlay) should be very fast to create as (in this example) we are basically creating a text file and we don't need to render images, extract text or provide natives.

 

A DAT file will be created with its delimiters for field and column separators. The review platform should be able to accommodate these delimiters when the overlay file is ingested. Below is a sample of what the overlay file should look like (I have opened this in Excel and removed the delimiters). You can see that I have metadata columns for the To, From, Subject, Primary date/time and File name fields.  You can also see that the Export set is included. This field may need to be renamed in the overlay file to match the field used for importing in the review platform.

image.png

Link to comment
Share on other sites

  • 1 year later...

I have a question about creating a load file.

One of the load file metadata fields I was instructed to incorporate is (per the ESI protocol doc):

ATTACH_RANGE

Description: Beginning and Ending Attachment numbers for parent and children. The number should be BEG_NO of the parent and END_NO for the last Child.

Looking at page 190/191 of Intella manual, this seems they want RECORD_ID_GROUP_BEGIN and then RECORD_ID_GROUP_END. If so, then how would I associate two Intella types to one new custom field "ATTACH_RANGE"?

 

Thanks.

Link to comment
Share on other sites

  • 1 year later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...