D1 file format change

Description

when were the " " removed fromeneratedD1 files?


History:

Original question from DOJ:
"Was there a recent change made to the File D1 format in the Broker (perhaps in the 11/30 release)?

It appears that double quotes (") around each value has been removed from the file format in the .csv generated in the Broker. I've attached a pull from 07/11/17 as well as one from this morning.

If the above is correct, are there more detailed release notes on what changed in the File D1 format? We use the D1 data for internal data quality checks and would need to document any differences."

Response:
The format change in the D1 file would be best linked to the release on 08/31/2017. This release updated the FPDS loader script to pull CSV extracts. More on this release here: https://broker.usaspending.gov/#/technicalHistory?_k=is6j07

Follow on question from DOJ:
Thank you. Can you confirm that the double quotes (") removal was part of this 8/31 update?

It doesn't specify below at the URL in the response:

"August 31, 2017

In this release, here is a list of technical changes that may require infrastructure or database updates, or represents additional functionality.

Alembic migrations:
Modified frec, cgac and subtieragency tables to accomodate FREC data
Added columns to detachedawardprocurement
Updated error messages for FABS validations
Updated SQL rules, including renaming 'D' validations to 'FABS' validations
Added new error types (rowCountError and fileTypeError)
Updated FABS derivations and FPDS script to accommodate FREC agencies
Updated FPDS loader script to pull csv extracts"

Assignee

Unassigned

Reporter

Lisa Mitchell

Labels

Service Desk Ticket Link

Epic Link

Priority

Low