As a downstream data user, I need fields that are not applicable in a given row to be labeled as such to help understand what's missing vs. N/A

Description

Background:
It's become apparent that many users of the data (e.g., GAO) assume that blank data = missing data. To help reduce confusion surrounding this, we could derive "N/A" in fields that we are 100% certain are blank for a given transaction rather than just missing. One risk of this approach is that people will be even more likely to assume blank = missing (though they seem to already do this). It also might take too much processing time and resources compared to the benefit. In the case of larger structural changes to D2 that differentiate and simplify it compared to FABS, #3 and #4 may be OBE by the time this story is considered.

In any case, if it is worth it, the lowest-hanging fruit here are A) IDVs vs. Awards in contract data (#1 and #2 below); B) place of performance in assistance data (#3 below); and C) Domestic vs. Foreign legal entities in Assistance data (#4 below).

1) IDVs: some fields are always blank for IDVs and some always blank for awards. Note that we want to do this conservatively (100% certainty). We never want to label something N/A that could actually just be missing.

2) Primary place of performance (PPoP) in assistance data can be single-zip, city-wide, county-wide, state-wide, or multi-state wide. When a PPoP is wide, fields that are more granular are not applicable. For example: a state-wide award would never have PPoPCounty or PPoP City or PPoP ZIP filled in. Yet we've seen users and GAO assume this means the data is missing for these awards. N/A would make this clearer.

A/C
1. Contracts: Refer to the latest version (probably v2.0) of IDD-D1. Filter by FPDS Award ATOM Feed Path column (Column K) using "[Not in Award XML atom Feed]" as the filter. These are fields that are 100% blank for award records in the atom feed (since the paths only exist in the IDV atom feed), so we should fill them in with N/A instead of blank.

2. Contracts: Refer to the latest version (probably v2.0) of IDD-D1. Filter by FPDS IDV ATOM Feed Path column (Column L) using "[Not in IDV XML atom Feed]" as the filter. These are fields that are 100% blank for IDV records in the atom feed (since the paths only exist in the award atom feed), so we should fill them in with N/A instead of blank.

3. Assistance: refer to the latest version (probably v2.0) of IDD-D2.
If PPoPCode is in 00FORGN format, then fill in N/A for
PrimaryPlaceOfPerformanceCityName
PrimaryPlaceOfPerformanceCountyCode
PrimaryPlaceOfPerformanceCountyName
PrimaryPlaceOfPerformanceStateName
PrimaryPlaceOfPerformanceZIP+4
PrimaryPlaceOfPerformanceCongressionalDistrict
If PPoPCode is in 00***** format, then fill in N/A for
PrimaryPlaceOfPerformanceCityName
PrimaryPlaceOfPerformanceCountyCode
PrimaryPlaceOfPerformanceCountyName
PrimaryPlaceOfPerformanceStateName
PrimaryPlaceOfPerformanceZIP+4
If PPoPCode is in XX***** format, then fill in N/A for
PrimaryPlaceOfPerformanceCityName
PrimaryPlaceOfPerformanceCountyCode
PrimaryPlaceOfPerformanceCountyName
PrimaryPlaceOfPerformanceZIP+4
If PPoPCode is in XX**### format, then fill in N/A for
PrimaryPlaceOfPerformanceCityName
PrimaryPlaceOfPerformanceZIP+4

4. Assistance:
If LegalEntityCountryCode = USA, fill in N/A for
LegalEntityForeignCityName
LegalEntityForeignProvinceName
LegalEntityForeignPostalCode
If LegalEntityCountryCode != USA, fill in N/A for
LegalEntityCityCode
LegalEntityCityName
LegalEntityStateCode
LegalEntityStateName
LegalEntityZIP5
LegalEntityZIPLast4
LegalEntityCountyCode
LegalEntityCountyName
LegalEntityCongressionalDistrict

Assignee

Unassigned

Reporter

Ross Williford

Labels

None

Service Desk Ticket Link

None

Priority

Medium