Skip Navigation

OURRstore: OU & Regional Research Store

Skip Side Navigation

OU & Regional Research Store (OURRstore)

The OU & Regional Research Store (OURRstore) is a very large scale tape archive storage instrument being deployed and maintained by the OU Supercomputing Center for Education & Research (OSCER) at the University of Oklahoma (OU).

OURRstore will be in production for 8+ years, and will have capacity for well over 100 Petabytes of tape (that is, well over 100,000 TB).

OURRstore will enable faculty, staff, postdocs, graduate students and undergraduates at the University of Oklahoma (OU), and at institutions across Oklahoma, as well as in other Great Plains Network (GPN) and Established Program to Stimulate Competitive Research (EPSCoR) states and territories, to build large and growing data collections, and to share these datasets with collaborators and others.

Via an innovative, low-cost business model, researchers buy their own tape cartridges, good for 8+ years, and pay zero usage charges (just tape cartridge and shipping costs).

OURRstore is expected to have hundreds of users.

OURRstore is intended for bulk data archiving, meaning small numbers of large files, not for zillions of tiny files. We can help you with that.

Please review OURRstore Policies and Procedures (below) for information regarding usage, best practices, costs and purchase process.

OURRstore is funded by a National Science Foundation (NSF) Major Research Instrumentation (MRI) grant (“Acquisition of a Regional Resource for Long-term Archiving of Large Scale Research Data Collections,” OAC-1828567, $967,755, 9/1/2018 - 8/31/2021, Principal Investigator Henry Neeman).

Interested in in using OURRstore? Contact OSCER at:

support@oscer.ou.edu

 

 

Table of Contents

SUMMARY
  Required Tape Format
  OURRstore Capacity
  File Sharing
WHAT IS OURRSTORE'S CURRENT STATUS
WHO IS ELIGIBLE TO BE AN OURRSTORE USER
WHAT KINDS OF DATA ARE ELIGIBLE TO BE ARCHIVED IN OURRSTORE
MEDIA: TAPE CARTRIDGES
HOW TO GET ACCESS TO OURRSTORE
HOW TO GET AN OSCER ACCOUNT
  OSCER Account Group Name
HOW TO GET OURRSTORE TRAINING
HOW TO PURCHASE MEDIA
FILE COMPRESSION
FILE SIZES
WRITE SPEED
RETRIEVE (READ) SPEED
REPLICATION FOR RESILIENCY
LIMITATIONS
USABLE FRACTION OF RAW TAPE CAPACITY
PURCHASING MEDIA: DETAILS
HARDWARE AND SOFTWARE SPECIFICATIONS

 

SUMMARY

 

 

OURRstore is intended for large scale (bulk) datasets being stored for the long term (several years).

OURRstore’s business model is:

  1. OU’s NSF Major Research Instrumentation grant (see above) pays for most of the hardware and software, along with initial warranties.
  2. OU’s Chief Information Officer sponsors space, power, cooling, world-facing network and labor, as well as annual support/maintenance of hardware and software after the initial warranties expire.
  3. Researchers purchase tape cartridges, at market price, to be placed into OURRstore, and pay shipping costs to get the tape cartridges to OU (and in the case of non-OU users, to ship some of the tape cartridges back to the owner).
 

Required Tape Format

OURRstore uses Linear Tape Open (LTO) tape cartridges, version 7 and later. We already support LTO-7 in “Type M” mode (explained below), and later on we’ll support LTO-8, LTO-9 and possibly LTO-10.

 

Currently Preferred LTO Version

Currently, the preferred LTO version is LTO-7 in “Type M” mode: 9 TB raw, ~7.65 TB usable per tape cartridge (~85% usable capacity). This is LTO’s lowest price per TB currently on the market.

 

Market Price

Market price of LTO-7 tape cartridges is currently ~$50.

In “Type M” mode, this corresponds to ~$6.50 per usable TB per copy, or $13 per usable TB for dual copies (REQUIRED for non-OU users, STRONGLY URGED for OU users), not including Indirect Costs (if purchased via external funds such as a grant).

NOTE: Tape cartridges are considered (consumable) materials, NOT EQUIPMENT, so they ARE subject to IDC.

 

OURRstore Capacity

OURRstore has room for a huge number of tape cartridges (initially ~11,000), and is expected to house over 100 PB (over 100,000 TB) of research data.

OURRstore’s first 824 tape cartridge slots are claimed by LTO-7 “Type M” tape cartridges holding files from the Oklahoma PetaStore, OU’s previous tape archive.

Of the remaining tape cartridge slots (over 10,000), half are reserved for OU users and the other half are available to all users, both halves on a first come, first served basis.

 

Prepaid Shipping of Tape Cartridges Containing Secondary Copies (Non-OU Users Only)

Non-OU users are REQUIRED to provide prepaid shipping labels or prepaid shipping boxes for their secondary copies to be shipped back to them at their home institutions.

(OU users’ secondary copies will be stored in an environmentally controlled facility at OU, remote from the tape library that holds their primary copies, so OU users DON’T have to provide prepaid shipping labels or boxes.)

 

File Sharing

Any OURRstore file owner can designate any of their OURRstore files and/or directories, via just a few mouse clicks, as downloadable by any specific researcher, or by any specific group of researchers, or by the whole world. (All files are non-shared by default.)

 

WHAT IS OURRSTORE’S CURRENT STATUS?

OURRstore is currently in “friendly user” mode.

Note that “friendly user” is the exact opposite of “user friendly:” OSCER reserves the right to change any aspect of OURRstore without notice (in practice, we’ll give sufficient notice, but we reserve the right not to).

We expect “full production” mode in fall 2021.

 

WHO IS ELIGIBLE TO BE AN OURRSTORE USER?

You may use OURRstore if you’re a researcher (non-academic professional, faculty, staff, postdoc, graduate student, undergraduate student, K-12) at any non-commercial research and/or education institution in any of the following categories:

  • at U Oklahoma (all campuses);
  • at any other non-commercial research and/or education institution in Oklahoma;
  • at any non-commercial research and/or education institution in any other Great Plains Network (GPN) full member state (Arkansas, Kansas, Missouri, Nebraska, South Dakota) – this ISN’T limited to full member institutions, but applies to any such institution in these states;
  • at any non-commercial research and/or education institution in any other Established Program to Stimulate Competitive Research (EPSCoR) jurisdiction (state or territory).

If you’re at a non-commercial research and/or education institution in a non-GPN/non-EPSCoR jurisdiction in the USA, you CANNOT get free access to OURRstore, but you CAN “buy in” to OURRstore, by purchasing a tape cartridge expansion cabinet and paying its annual support. If you’re interested in that option, please contact:

support@oscer.ou.edu

Please note that, because of data center floor space limitations, there are a VERY LIMITED number of such expansion cabinet opportunities available.

You MUST be an employee of, or a student at, an eligible institution (as described above), to be eligible to use OURRstore.

If you’re at a commercial firm, and/or if you’re an employee of, or a student at, any institution outside the USA, then you CANNOT use OURRstore AT ALL.

 

WHAT KINDS OF DATA ARE ELIGIBLE TO BE ARCHIVED IN OURRSTORE?

OURRstore is for Science, Technology, Engineering & Mathematics (STEM) research data ONLY, because OURRstore is funded by an NSF grant.

OURRstore is NOT for clinical research data, NOR for non-STEM data, likewise because OURRstore is funded by an NSF grant.

NOTE: The NSF’s definition of STEM includes physical sciences, biosciences, geosciences, engineering, mathematics, technology (for example, computer and information sciences), and social sciences.

OURRstore is intended for NON-CLINICAL STEM RESEARCH DATA that is NOT LEGALLY REGULATED.

Non-STEM data is currently FORBIDDEN on OURRstore, because OURRstore was funded by the NSF, and non-STEM data is outside the NSF’s mandate. (EXCEPTION: All data from the Oklahoma PetaStore will be copied to OURRstore, and that might include non-STEM data.)

Clinical data (for example, clinical studies of human disease) is currently FORBIDDEN on OURRstore, because OURRstore was funded by the NSF, and clinical data is outside the NSF’s mandate.

Legally regulated data (for example, HIPAA, Controlled Unclassified Information, ITAR/EAR) is currently FORBIDDEN on OURRstore, because it’s outside the stipulations in OU’s NSF grant text.

We ALLOW human subjects data that isn’t in any of the forbidden categories listed above. If your files are subject to one or more Institutional Review Board (IRB) agreement(s) governing human subjects research, whether with OU’s IRB and/or with another institution’s IRB, then it’s YOUR RESPONSIBILITY to ensure full compliance with your IRB agreement(s).

If you’re at OU and you have legally regulated data, please contact:

support@oscer.ou.edu

We’re working on a solution for OU users only, but it’s still in early stages.

If you’re at another institution, we DON’T expect to have a solution for you for legally regulated data, NOR for clinical data, NOR for non-STEM data.

 

MEDIA: TAPE CARTRIDGES

OURRstore has room for a very large number of LTO tape cartridges (currently ~11,000 LTO tape cartridge slots total, expandable to ~15,000 LTO slots, at no charge to you).

Each LTO-7 Type M tape cartridge holds up to 9 TB raw, of which ~85% is usable in practice (~7.65 TB usable).

We strongly urge you to keep at least dual copies of every file, on two different pieces of media – so either buy twice as many tape cartridges, or store the second copy elsewhere than OURRstore (for example, at one of the national supercomputing centers).

NOTE: Non-OU users of OURRstore MUST keep at least dual tape copies, in accordance with the terms of our NSF grant. OU users of OURRstore are STRONGLY URGED to do so.

 

HOW TO GET ACCESS TO OURRSTORE?

Before you can use OURRstore, you MUST do ALL of the following:

  1. Get an OSCER account, if you don’t already have one (see below).
  2. Complete, sign and submit a Use Agreement form, as follows:
            By e-mail (STRONGLY PREFERRED): support@oscer.ou.edu
            By fax: 405-325-5486
            By OU campus mail: Engineering Lab 212
            By postal mail: Henry Neeman, Engineering Lab 212, 200 Felgar St, Norman OK 73019
    Once we receive your form, we’ll acknowledge it so that you know you can start moving forward.
    NOTE: We may have a new Use Agreement form later, in which case you’d need to do that one as well.
  3. Purchase at least 2 tape cartridges and have them shipped to OU (see below). You DON’T need to wait for the tape cartridges you’ve purchased to arrive at OU to complete step 4, below, and then to start using OURRstore.
  4. Complete an OURRstore training session (~1 hour; see below).

 

HOW TO GET AN OSCER ACCOUNT?

If you don’t yet have an OSCER account, you can request an OSCER account at:

http://www.oscer.ou.edu/account_request/

In your OSCER account request, in your research description, please first state that you plan to use OURRstore.

If you’re from OU, or from another Oklahoma non-commercial research and/or education institution, or you’re actively collaborating on a research and/or education project that has an Oklahoma Principal or Co-Principal Investigator, then you can get a regular OSCER account that includes supercomputer access in addition to OURRstore access.

Otherwise, you’ll get an OURRstore-only OSCER account.

 

OSCER Account Group Name

If you AREN’T the Principal Investigator (for example, lead faculty member) of your research team, then please consult with your research team’s PI to find out what research group name you should use.

If you ARE the PI of your research team, please pick a research group name as follows:

  • 3 to 12 characters long, lower case letters and digits only, first character a lower case letter.
  • DON’T use a research group name based on the name OURRstore, but instead choose a name based on the research your team does.
  • Non-OU groups: Please prepend your institution’s abbreviation at the start of your group name. For example:
            kublahblah   (U Kansas)
            ecublahblah  (East Central U)
            uamsblahblah (U Arkansas Medical Sciences)
    NOTE: OU research teams DON’T need to prepend ou to their group name.
  • For the rest of the characters in your group name (up to 12 characters total, including the prepended institution abbreviation), if you aren’t sure what to name your research team’s group:
    • Describe your research team’s topic area in 10 words or fewer.
    • Distill that phrase down to the right number of characters.
              For example, if you’re at U Nebraska Omaha and your research topic is Colliding black holes then your group name could be something like this:
                  unocolblkhol
              Which perfectly fits in the 12-character limit!
  • If you’re the PI of your research team and you want to name your research team based on your name, you can do that. In which case, please use your last/family name, followed by your first/given name or initial, if there’s room. (If you have a long family name, you might have to truncate it and skip the first initial.)
            Examples:
                umkcleekim
                ksuhorvathw
                ualrmelendez
                uvmfitzgeral

 

HOW TO GET OURRSTORE TRAINING?

Once your OSCER account has been created, AND we’ve received your completed Use Agreement form, AND you’ve purchased your research group’s media (tape cartridges), THEN you can schedule an OURRstore training session with the OSCER Operations Team, to train you and/or your research team on how to use OURRstore properly.

To schedule your OURRstore training, please contact us at:

support@oscer.ou.edu

The training might be in person, or remotely via videoconferencing.

Some portion of the OURRstore training might be via written documentation and/or recorded video(s).

NOTE: You may take the training after you have purchased tape cartridges but before those tape cartridges have been delivered to OU.

 

HOW TO PURCHASE MEDIA (LTO TAPE CARTRIDGES) FOR OURRSTORE?

Purchases MUST be of the approved brand (IBM) and one of the approved model numbers.

For non-OU users of OURRstore, purchases can be from any reseller, subject to whatever rules your home institution imposes. (For example, some institutions require sole source justification or multiple quotes for purchases above a specific threshold dollar amount.)

For OU users of OURRstore, purchases below $5000 can be from any reseller, but purchases $5000 and up are subject to restrictions (see below).

WARNINGDON’T purchase ANY tape cartridges until you’ve consulted us first! Our warranty coverage FORBIDS the use of unapproved tape cartridge brands or model numbers, so any unapproved tape cartridges will be THROWN AWAY, or shipped back to you if you pay for shipping under the terms described below.

We’ll be happy to help you with your LTO tape cartridge purchase. Please contact us at:

support@oscer.ou.edu

for current pricing and further assistance.

Please also note that, because tape cartridges are classified as “materials” instead of “permanent equipment,” then if you’re buying the tape cartridge(s) via external funds (for example, a grant), then your purchase is subject to Indirect Costs (IDC), also known as “Facilities & Administration” or “Overhead.”

For OU external funds, the IDC rate can be found here. (Look for the “Organized Research” rate.)

Other institutions typically publish their IDC rates on their Vice President/Vice Chancellor/Vice Provost for Research website.

For help with purchasing tape cartridges, please contact us at:

support@oscer.ou.edu

We’ll help you coordinate with the appropriate reseller(s) to get a quote and execute the purchase.

Details about purchasing can be found below.

 

DON’T BUY TAPE CARTRIDGE BARCODE LABELS

OU IT buys tape cartridge barcode labels in bulk, using our preferred numbering scheme.

So if you also buy barcode labels, we’ll THROW THEM AWAY.

 

OURRSTORE IS FOR ARCHIVING, NOT FOR BACKUPS

OURRstore is for ARCHIVING ONLY – NO BACKUPS.

Here, “backup” means “frequently make copies of all new or recently changed files (‘incremental’), and infrequently make copies of all files (‘full dump’).”

In a backup, deleting a file from the source disk that’s being backed up (eventually) deletes the same file from the backup copy.

Whereas, “archive” means “Write Once, Read Seldom.”

In an archive, deleting a file from the source disk that’s being archived NEVER deletes the same file from the archive. In order to delete the archived copies of the file, you’d have to log in to the archive (for example, OURRstore) and delete the archived copies of that file manually.

 

FILE COMPRESSION

To save both (a) tape space (and therefore your money) and (b) disk space on the disk you’d retrieve your files to, we strongly urge you to pre-compress all your files.

We can help you learn how to do that. Contact us at:

support@oscer.ou.edu

For example, you can use gzip or Zip to pre-compress your files, before copying them to OURRstore.

 

FILE SIZES

Please note that OURRstore CANNOT be used for small files (smaller than 1 GB per file).

We strongly urge that all files stored on OURRstore be between 20 GB and 200 GB, although in principle you can store files as small as 1 GB or as large as 1 TB.

Each file being retrieved takes 30 to 120 seconds of preparation time (typically ~1 minute) before the relevant tape drive can read or write the relevant portion of the relevant tape cartridge:

  1. The tape cartridge that was already in the relevant tape drive might need to be rewound or fast forwarded to a different position.
  2. That tape cartridge is ejected from its tape drive.
  3. The tape robot collects that tape cartridge and carries it to its tape cartridge slot.
  4. The tape robot collects, from its tape cartridge slot, the tape cartridge that holds/will hold your file(s).
  5. The tape robot puts your tape cartridge into the relevant tape drive.
  6. The tape drive rewinds or fast forwards your tape cartridge to the appropriate position.

Therefore:

1     GB file: ~1 minute prep time (on average) + ~3 seconds   read or write time – Prep time dominates!

20   GB file: ~1 minute prep time (on average) + ~1 minute    read or write time – Prep time ~ read time

200 GB file: ~1 minute prep time (on average) + ~11 minutes read or write time – Read time dominates!

1     TB file: ~1 minute prep time (on average) + ~1 hour         read or write time – Read time dominates!

So, on the one hand, we don’t recommend files smaller than 20 GB, because far more time will be spent preparing to read or write the file than actually reading or writing it.

But, on the other hand, we don’t recommend files larger than 200 GB, because of long retrieval times (up to ~1 hour).

If you absolutely need to have file(s) of greater than 200 GB, please contact to us about that:

support@oscer.ou.edu

If you have many small files (under 1 GB each), then you can aggregate them into a Zip file or a gzipped tar file (Unix equivalent of a Zip file), in order to achieve the minimum acceptable file size for OURRstore.

We can help you learn how to do that:

support@oscer.ou.edu

If you have individual files of more than 1 GB each, we strongly recommend compressing them via Zip or gzip (or other comparable file compression methods), even if you aren’t aggregating multiple files into a single Zip or tar file.

We can help you learn how to do any of these tasks:

support@oscer.ou.edu

You’ll need to create the Zip file or gzipped tar file on the source disk system where the files are coming from, NOT on OURRstore.

That is, the files that you want to upload to OURRstore should ALREADY be compressed and/or converted to Zip or tar files, before uploading them to OURRstore.

Exception:

In the near future, we’ll create a script that will automatically detect files in specific directories that are to be archived, and will then automatically create the appropriate tar or Zip file(s) and do the appropriate compression for you.

If and when that’s available at your institution, that’ll be even better than doing those tasks by hand.

 

WRITE SPEED

We’ve benchmarked file store (write) speeds into OURRstore as follows:

  • ~X GB/sec from /scratch on OSCER’s current supercomputer, Sooner, for a non-parallel copy (1 TB should be ~X hours);
  • ~X GB/sec from Sooner’s /scratch, for a parallel copy (1 TB should be ~X hours);
  • ~X GB/sec from OURdisk (1 TB should be ~X hours) – OURdisk is OSCER’s current “condominium” disk offering;
  • ~X GB/sec via Globus from a large scale, high performance filesystem outside OSCER;
  • ~X GB/sec via sftp or scp from a large scale, high performance filesystem outside OSCER.

Note that uploading a file from a laptop/desktop PC will be slower than the above, as will uploading a file from a handheld (phone, tablet) – the bottleneck will be the PC or handheld.

NOTE: Because file uploads go to OURRstore’s disk instead of directly to OURRstore’s tape, your file uploads won’t spend any time waiting for your store-to-tape operation to get started; instead, that happens invisibly behind the scenes after your file upload completes, typically within ~X hours.

 

RETRIEVE (READ) SPEED

We anticipate file retrieval (read) speeds to be roughly the following:

  • time spent waiting for a tape drive to become available, which will depend entirely on how many other file transfers are ongoing or waiting to begin,

PLUS

  • time spent on preparation before reading (see above),

PLUS

PLUS

  • copying your file(s) back to wherever you actually want to download the file to (see above).

Thus, other than time spent waiting for a tape drive to become available, total retrieval time should be:

  • 1 GB file:   ~1 minute;
  • 20 GB file:   ~2 minutes;
  • 200 GB file: ~12 minutes;
  • 1000 GB file:   ~1 hour.

Note that, at least for the near future, your time spent waiting for a retrieval to get started should be very modest in most cases.

This is because OURRstore has 6 LTO-8 tape drives and gives priority to file retrievals (reads) when there are a mix of stores and retrieves waiting to get started.

 

REPLICATION FOR RESILIENCY

Tape cartridges can break (and have done so). Therefore, you need MULTIPLE copies of your archived files, on MULTIPLE distinct tape cartridges.

You can choose which of the following replication policies to use, on a per-file or per-directory basis.

Non-OU users:

You’ll be able to store your files under either of the following replication policies:

  • tape_2copies: One copy inside OURRstore’s tape library, one copy shipped back to you, using the prepaid shipping label that you provided us.

The disadvantage of tape_2copies is, if the primary copy in the OURRstore tape library fails (for example, if the tape breaks), then you’d have to ship your secondary copy to OU, which might be the ONLY REMAINING COPY OF THOSE FILES IN THE WORLD, so you’d be taking a BIG RISK in shipping it back to us.

  • tape_3copies: One copy inside OURRstore’s tape library, one copy shipped back to you, and one copy carried by us to another OU campus to an environmentally controlled facility for offsite storage.

This costs 50% more than tape_2copies, but if the primary copy inside the OURRstore tape library fails, we can quickly restore your files from the tertiary copy located elsewhere at OU, instead of you having to ship us your secondary copy, so your RISK IS REDUCED A LOT.

OU users:

You’ll be able to store your files under either of the following replication policies:

  • tape_2copies: One copy inside OURRstore’s tape library, one copy carried by us to another OU campus, to an environmentally controlled facility for offsite storage – we expect this to be the MOST POPULAR REPLICATION POLICY BY FAR for OU users.
  • tape_3copies: One copy inside OURRstore’s tape library, two copies carried by us to another OU campus, to an environmentally controlled facility for offsite storage, so this costs 50% more than tape_2copies, with no major advantage other than ensuring that there are at least 3 copies total.

OU users only will have additional options, which are available ON REQUEST ONLY:

  • disk_1copy_tape_1copy, if you’ve purchased disk capacity (EXPENSIVE!);
  • disk_1copy_unsafe, if you’ve purchased disk capacity (EXPENSIVE!), for files that are available at other repositories (for example, at a national data repository);
  • tape_1copy_unsafe, for files that are also available at other repositories (for example, at a national data repository).

NOTE to OU users: You can only choose a particular replication policy if you’ve purchased sufficient appropriate media (tape cartridges and/or disk drives) for it.

For example, if you only own tape cartridges but not disk drives, then you can choose options that involve tape only, not options that involve both tape and disk, nor options that involve disk only.

If you owned disk on the Oklahoma PetaStore, that disk ownership ENDED when the PetaStore was decommissioned, so you’d have to buy new disk for OURRstore.

Please note that we DON’T allow multiple copies on disk, because of the severe scarcity and limited expandability of disk capacity.

 

LIMITATIONS

Please carefully read the Use Agreement form to understand the limitations of using OURRstore, including the kinds of files that CANNOT be stored on OURRstore, who CANNOT use OURRstore, and YOUR RESPONSIBILITIES when using OURRstore.

3, 2, 1 RULE

The 3, 2, 1 Rule recommends storing your files as follows:

  • at least 3 copies of each file;
  • at least 2 different media types (for example, disk and tape);
  • at least 1 copy remote from the others.

If you want to follow the 3, 2, 1 Rule, you can achieve that via a combination of (a) OURRstore and (b) local disk at your institution.

Examples:

Non-OU Users: If you upload a file to OURRstore from disk at your local institution (for example, on a PC or on your local supercomputer), and you leave the file on disk at your local institution, then you’ll have one copy on your local institution’s disk, one copy inside OURRstore’s tape library at OU, and one copy shipped back to you at your local institution. So you’ll have 3 copies total, on two different media types (disk, tape), with one copy remote (at OU inside OURRstore).

OU Users: If you upload a file to OURRstore from any of (a) your local disk (for example, on your own PC), or (b) OSCER’s supercomputer, or (c) {OURdisk[URL to come]}, and you leave the file on disk, then you’ll have one copy on disk, one copy inside OURRstore’s tape library at OU Norman, and one copy in an environmentally controlled facility at OU Health Sciences Center in Oklahoma City (~24 miles from the OURRstore tape library). So you’ll have 3 copies total, on two different media types (disk, tape), with one copy remote (at a different OU campus).

NOTE: Not every research project needs to follow the 3, 2, 1 Rule, and there aren’t good rules of thumb for deciding whether your research project does need to, so use your project’s Data Management Plan (if you have one) as a guide.

 

USABLE FRACTION OF RAW TAPE CAPACITY: ~85%

LTO tape cartridge capacities are as follows:

  • LTO-7:
    6 TB raw,   ~5.1 TB usable (85%) – NOT OFFERED (Why waste space?)
  • LTO-7 Type M:
    9 TB raw,   ~7.6 TB usable (85%) – currently the best $/TB
  • LTO-8:
    12 TB raw, ~10.2 TB usable (85%) – expected to be breakeven in $/TB c. 2022
  • LTO-9:
    18 TB raw, ~15.3 TB usable (85%) – to be released in late 2021, breakeven $/TB c. 2025[NHJ3]

LTO-7 Type M

LTO-7 Type M is just a regular LTO-7 tape cartridge, but formatted in a special way, using a special barcode label (which OU provides at no charge), and written/read by an LTO-8 tape drive (which OURRstore has several of).

So LTO-7 Type M at 9 TB raw/~7.65 TB usable has IDENTICAL PRICE to regular LTO-7 at 6 TB raw/~5.1 TB usable. Therefore, OURRstore ENCOURAGES LTO-7 Type M (for now) and FORBIDS regular LTO-7.

We recommend budgeting as if each LTO tape cartridge has 85% of the raw space available to use, in case of the following issues:

  • no files are available that are small enough to fit in the last 15% of the tape cartridge;
  • contingency, because we can’t guarantee that any particular tape cartridge will be 100% filled;
  • unanticipated growth of your datasets.

In addition, you’ll want to think about whether your files will, over the long term, reside ONLY on OURRstore, or ALSO on some other long-term archive.

If these files will be stored ONLY on OURRstore, then we urge you to keep at least dual copies of all files on separate tape cartridges, which doubles the number of tape cartridges. (Non-OU users of OURRstore are REQUIRED to do this, by the terms of the OURRstore NSF grant.)

 

PURCHASING MEDIA: DETAILS

Indirect Costs

Tape cartridges purchased on external funds such as a grant typically ARE subject to Indirect Costs (also known as “Facilities & Administration” or “Overhead”), because they’re considered consumable materials, not “permanent equipment.”

OU users: OU’s current IDC rate can be found here. Click on Download signed original agreement (PDF) and look for the current On Campus Organized Research rate (or whichever rate category is appropriate for your funding, if you know that it’s a different rate category).

Non-OU users: Other institutions’ IDC rates are typically found on the website of their local Vice President/Vice Chancellor/Vice Provost for Research.

 

Tape Cartridge Make and Models Allowed in OURRstore

OURRstore is allowed to have ONLY IBM-branded LTO tape cartridges, of ONLY the following models:

  • IBM 38L7302 [LTO-7, including Type M]
  • IBM 01PL041 [LTO-8]

Any other tape cartridge brands, or any other IBM model numbers, are ABSOLUTELY FORBIDDEN in OURRstore, because they would risk voiding our warranty/support/maintenance agreements.

Some tape cartridge resellers also sell multi-packs (usually 5-packs and/or 20-packs), sometimes at a lower price per tape cartridge than buying large numbers of individual cartridges. If you want to purchase such multi-packs, you MUST confirm with the reseller, in writing, that the individual tape cartridges in the multi-pack are one of the above model numbers.

 

Shipping Address

ALL OURRstore tape cartridges, for both primary and secondary copies, should be shipped to:

Shane Smith, OURRstore, YOURGIVENNAME YOURFAMILYNAME
301 David L. Boren Blvd. Loading Dock 1028
Norman OK 73072
405-255-4308
smsmith@ou.edu
        (but replacing YOURGIVENNAME with your first/given name and YOURFAMILYNAME with your last/family name).

By putting your name on the first line of the shipping address, we’ll know who these tape cartridges belong to (we receive many such shipments).

If you’re using a quote for the tape cartridges, please have the reseller put exactly the same shipping address in the quote, because some resellers only look at the shipping address in the quote, instead of the shipping address in the Purchase Order.

 

Tape Cartridge Resellers

You can purchase these tape cartridges from ANY RESELLER you want, subject to the caveat below for OU users.

Using your favorite search engine (for example, Google, DuckDuckGo, Bing, etc), search for:

IBM 38L7302

OR

IBM 01PL041

Note that we don’t yet recommend the latter, because LTO-8 is still more expensive per TB, but if you insist on buying LTO-8 tape cartridges, OURRstore will support them.

But, when LTO-9 tape cartridges become available, not only will be they be too expensive per TB, OURRstore also won’t be able to support them, because we don’t yet have LTO-9 tape drives to read and write LTO-9 tape cartridges.

From the web search above, you’ll see that there are several resellers of LTO tape cartridges.

OU users:

Tape cartridge purchases should be executed by your department’s/unit’s financial lead.

If you’re buying tape cartridges that total less than $5000, your department’s/unit’s financial lead can do a “Pcard” purchase. You DON’T need multiple quotes for that, nor a quote – your unit’s financial lead can purchase directly from a reseller’s website if they want.

If you’re buying enough tape cartridges to hit $5000 or more in a single purchase, then:

  • You MUST get at least 3 quotes, from 3 different resellers, all for the same brand, model and quantity of tape cartridges. OSCER can help with that: support@oscer.ou.edu
  • You MUST pick the least expensive quote, unless you have a compelling reason to do otherwise.
  • Your department’s/unit’s financial lead must enter the purchase into OU’s internal purchasing system, “PeopleSoft Marketplace,” using the shipping address above and your relevant OU account (for example, a grant account).
  • After your department’s/unit’s financial lead has submitted the purchase, OU Procurement can send a Purchase Order to the selected reseller.

Once tape cartridges are purchased, there AREN’T any recurring charges (for example, NO monthly service charge).

But, at OURRstore’s end-of-life, you might need to buy new tape cartridges (though we hope that won’t be the case).

Based on the current rate of tape cartridge capacity increase (typically doubling roughly every 2½ years), and assuming a 3% inflation rate per year and no change in the IDC rate, the cost in 8 years will be ~15% of today’s cost for the same usable capacity.

Obviously, there’s no way to know (let alone guarantee) that these rates will remain constant, so consider these numbers to be best guess estimates.

NOTE: We plan to run a Request for Quotes from as many LTO tape cartridge resellers as we can identify, to get a long-term commitment on tape cartridge pricing and/or discount rates, so that 3 quotes won’t be needed for purchases above $5000. We’ll update when that’s done.

 

DON’T Buy Barcode Labels

OU IT provides barcode labels at no charge, so that our barcodes have the correct format (especially for LTO-7 Type M) and the correct numbering order.

Please DON’T buy your own barcode labels, regardless of which institution you’re at.

 

Pre-formatted Tape Cartridges

Please DON’T pay for pre-formatting of tape cartridges, unless you’ve contacted us first: support@oscer.ou.edu

We’ll need to confirm with the tape cartridge reseller that the pre-formatting is of the correct type (for example, LTO-7 Type M pre-formatting instead of LTO-7 pre-formatting).

 

NON-OU USERS: PREPAID SHIPPING OF SECONDARY CARTRIDGES TO YOU

Non-OU users MUST provide prepaid shipping boxes and/or labels to OU, to ship your secondary tape cartridges back to you, preferably at your home institution.

(OU users DON’T need to do this.)

 

Shipping Weight and Dimensions for LTO Tape Cartridges

Without plastic “jewel cases”:

  • LTO tape cartridge shipping weight, without jewel case, each: 0.441 lbs / 200 g
  • LTO tape cartridge size, without jewel case, each: 4 in × 4.1 in × 0.8 in / 10.2 cm × 10.5 cm × 2.1 cm

With “jewel cases”:

  • LTO tape cartridge shipping weight, with jewel case, each: 0.6 lbs / 270 g
  • LTO tape cartridge size, with jewel case, each: 43/8 in × 41/2  in × 11/8 in / 11.1 cm × 11.4 cm × 2.9 cm

 

Prepaid Return Shipping Boxes or Shipping Labels for Non-OU Users

Please purchase one or more prepaid return shipping boxes or labels for the tape cartridges that contain secondary copies of files, which will be shipped back to you, preferably at your home institution.

Those can be sent to us physically (prepaid cardboard boxes and/or prepaid paper shipping labels), or electronically (for example, prepaid PDF shipping labels).

If you have to send us cardboard or paper instead of electronic, please alert us about that, so that we know to look for it.

Alternatively, you can wait until we’re ready to send you the tape cartridges, but the risk there is that your grant will expire (or run out of money) before you make the prepaid shipping box or label purchase.

You’re allowed to use any shipping carrier you want that can:

  1. provide a prepaid cardboard shipping box or prepaid paper or electronic shipping label (PDF preferred for electronic),
    AND
  2. schedule a pickup from the Four Partners Place data center at OU’s Norman campus, using the same address as below (this must be scheduled with the OSCER Operations Team).

NOTE: OU WON’T pay for shipping boxes and/or shipping labels of tape cartridges for ANY non-OU user, NOR will OU transport tape cartridges to a shipping carrier’s office or dropoff location, even if that shipping carrier has a dropoff location elsewhere on the OU Norman campus – we are NOT A TAXI SERVICE.

 

US Postal Service Prepaid Forever Priority Mail

An example of a service that meets our criteria is the US Postal Service’s Prepaid Forever Priority Mail (2-day), with the following pricing (as of May 20 2021):

Qty (tapes)            Price per box                        Prepaid Forever Flat Rate Box Type

1                                $8.45                                     Small          (  5 3/8   inch ×   8 5/8 inch × 1 5/8 inch)

2-12                          $15.50                                    Medium #2 (13 5/8    inch × 11 7/8 inch × 3 3/8 inch)

13-20                        $21.90                                    Large          (23 11/16 inch × 11 3/4 inch × 3 inch)

The US Postal Service will only sell these boxes in minimum lots of 5 of the same box size, so the minimum total shipping cost per box purchase is $42.25 for 5 × Small, or $77.50 for 5 × Medium, or $109.50 for 5 × Large.

Thus, the shipping price per tape cartridge varies from $8.45 each to $1.10 each, depending on how many are being shipped at a time.

If you expect to ever have at most 5 tape cartridges for OU to ship back to you, then you may be better served buying a 5-pack of Small boxes and having us ship your tape cartridges one per box, because the cheapest possible purchase is 5 Small boxes at $42.25 total.

If you expect to continue buying tape cartridges over OURRstore’s 8+ year lifetime, then you should buy 5 Large or 5 Medium boxes at a time, as needed.

(DON’T buy US Postal Service prepaid envelopes, only prepaid boxes, because we can’t put any padding in envelopes.)

The US Postal Service requires you to provide a return address for your Prepaid Forever Flat Rate Boxes, which in this case would be OU (which will use these boxes for shipping your tape cartridges to you). Please use:

Shane Smith
University of Oklahoma OURRstore YOURGIVENNAME YOURFAMILYNAME
301 DAVID L BOREN BLVD
STE 1000
NORMAN, OK 73072-7341
(Except replace YOURGIVENNAME with your given/first name, and replace YOURFAMILYNAME with your family/last name. And NO PUNCTUATION in the line that says University of Oklahoma. If you run out of room on that line, you may shorten the university’s name to “U Oklahoma,” or drop OURRstore, or shorten your name.)

That will cause the US Postal Service to ship these empty boxes (with prepaid shipping labels on them) to OU.

Then, e-mail us how many boxes of which sizes you’ve shipped to us, along with the address to ship your secondary tape cartridges to (preferably at your home institution).

Note that this return address is DIFFERENT from the address to have your tape cartridge reseller ship the tape cartridges to, above. (We tested the other shipping address, the one for buying tape cartridges, but the US Postal Service website won’t accept it, only this return address.)

If you want to use another carrier, you’ll need to have that carrier deliver their packing boxes to OU, and to provide OU with a prepaid shipping label or prepaid shipping box, and to schedule a pickup at OU, all at no charge to OU.

NOTE: In the context of OU shipping tape cartridges, the quantity of tape cartridges is the quantity to be shipped back to your institution (that is, either one half or one third of the total quantity of tape cartridges you’ve purchased, depending on whether you’re using tape_2copies or tape_3copies, respectively).

 

HARDWARE AND SOFTWARE SPECIFICATIONS

Tape Library: IBM TS4500

Maximum capacity, usable, as currently configured: 83.7 PB (LTO-7 Type M, 10,940 tape cartridge slots in 9 frames)

Maximum capacity, usable, maximal system: 114 PB (LTO-7 Type M, 14,900 tape cartridge slots in 12 frames)

Current slot capacity: 10,940 tape cartridge slots, 28 tape drive slots

Tape drives, current: 6 × LTO-8 (can read and write LTO-8, LTO-7 Type M, LTO-7)

Tape drives, future: 4 to 6 × LTO-9 or LTO-10 (can read and write LTO-9, maybe LTO-10)

Frames: 1 controller frame (L55), 1 drive/cartridge expansion frame (D55), initially 7 cartridge-only expansion frames (S55), will add 1 more S55 frame plus 2 S54 expansion frames upconverted from OSCER’s old IBM TS3500 tape library

Tape cartridge format, current: LTO-7 Type M

Tape cartridge formats, future: LTO-8, LTO-9, MAYBE LTO-10

Tape cartridge capacity, each, uncompressed:

  • LTO-7 Type M:
    each   9 TB raw, ~7.65 TB usable
  • LTO-8:
    each 12 TB raw, ~10.2 TB usable
  • LTO-9:
    each 18 TB raw, ~15.3 TB usable
  • LTO-10:
    each 36 TB raw, ~30.6 TB usable – MAYBE

Tape drive speed, uncompressed

  • LTO-8 drive for LTO-7 Type M cartridge:
    300 MB/sec peak per LTO-8 drive
  • LTO-8 drive for LTO-8 cartridge:
    360 MB/sec peak per LTO-8 drive
  • LTO-9 drive for LTO-9 cartridge:
    400 MB/sec peak per LTO-9 drive
  • LTO-10 drive for LTO-10 cartridge:
    UNKNOWN – MAYBE

 

Disk Subsystem (as front-end “landing pad” for files coming on and off tape): IBM FlashSystem 5030

  • Disk drive format: Nearline SAS 7200 RPM
  • Disk drive capacity, each, raw: 14 TB
  • Current number of drives: 60
  • Current capacity: 456 TB usable
  • Enclosures: One Large Form Factor (3.5-inch) Control Enclosure (12 drive bays), 4 LFF Expansion Enclosures (12 drive bays each)
  • Network, Fibre Channel: 8 ports 16 Gbps

 

Servers, Tape Control: IBM LC922 model 22P

  • CPUs: dual POWER9 16-core
  • RAM: 128 GB (8 × 16 GB) DDR4
  • Disk: quad Nearline SAS 2 TB 7200 RPM
  • Network, Ethernet: Intel XL710 10GE SFP+ quad-port with transceivers
  • Network, Fibre Channel: QLogic QLE2692SR dual-port 16 Gbps
  • Operating System: Red Hat Enterprise Linux

 

Servers, Disk Control: Lenovo ThinkSystem SR650

  • CPUs: Intel Xeon Cascade Lake 4214 12-core 2.2 GHz
  • RAM: 192 GB (12 × 16 GB), DDR4 2933 MHz
  • Disk: dual SAS SSD 800 GB PM1645a
  • Network, Ethernet: Lenovo 10GE SFP+ quad-port with dual transceivers, Intel i350 GigE dual-port
  • Network, Fibre Channel: QLogic QLE2692SR dual-port 16 Gbps
  • Operating System: Red Hat Enterprise Linux

 

Switches, Fibre Channel: IBM SAN24B-6

  • 24 Fibre Channel 32 Gbps ports per switch, 11 Fibre Channel 16 Gbps transceivers per switch

 

Switches, Ethernet:

  • Dell S4128F-ON: 2 switches, each 28 10GE SFP+ ports, 2 100GE QSFP28 ports, connected/failover via Virtual Link Trunking
  • Dell N3048ET-ON: 2 switches, each 48 GigE RJ45 ports, 2 10GE SFP+ ports, connected/failover via stacking

 

Software, Tape Control: IBM Spectrum Archive

 

Software, Disk Control: IBM Spectrum Scale (formerly GPFS) Standard Edition

 

INTERESTED IN USING OURRSTORE?

Contact OSCER at: support@oscer.ou.edu