Skip to content
#

data-profiling

Here are 28 public repositories matching this topic...

PSUlion16
PSUlion16 commented Apr 9, 2020

As a user,

It would be nice to have the "Observed Value" Field be standardized to show percentages of "successful" validations, vs a mix of 0% / 100%. This causes confusion as there are different levels of validation outputs with different verbage (making someone not used to the expectations confused) I've given an example below in a screenshot for what I mean:

![image](https://user-images.g

The program compares two files at a time and does the following 1.Gathering metadata on the individual tables(column count,record count,list of columns with datatype etc) 2.Identifying matching columns between tables based on names as well as data. Using machine learning, we are handling syntactic as well as semantic variations of column names for accurate matching. 3. Finding duplicate columns in single table with the option to deduplicate if required 4. Finding columns with missing data/null values.

  • Updated Feb 17, 2018
  • Python

Identified data types for each distinct column value on 1900 data sets. For each column, summarized semantic types present in the column, using Fuzzy Logic, Levenshtein distance. Identified & derived inference the 3 most frequent 311 complaint types by borough.

  • Updated Apr 15, 2020
  • Python

Improve this page

Add a description, image, and links to the data-profiling topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with the data-profiling topic, visit your repo's landing page and select "manage topics."

Learn more

You can’t perform that action at this time.