Its neat and tidy is what we would like to see in our rooms. The same applies to data we work in our Business.
If we have shirts, all shirts can be grouped and arranged in one manner and if we have books, all books of same size can be arranged in a similar manner. This organizing skills play a major role in data science labs. The data scientists need that organizing pattern to arrange the data for clean, crisp, neat and tidy visualization or presentation of data. The basic of organizing is understanding the rows and columns. The same organizing principle of how to arrange better in the rack applies to the data world as well.
1. Standardize things when we have more than one people working on it. This simply avoids clutter.
The Column heading ideally represent the variables such as Name, Age, Company and other details of the person for example. The Rows have the data for the headings and the row can be kept adding in time. If we alter the column down the line, the data might gets messy if not taken proper care. This requires an organizing mind to plan and work ahead on the need basis.
In another case, if it requires that the data has been collected from different branches or from different sources, then it might be messy and it requires cleaning in hand. Other messy things commonly found in Business data is the Name which has first name and last name included in some cases and first name only in some cases. Also the address field may represent both city and state details together and sometimes it becomes difficult if we just need to see how many people have cars in a particular state.
2. The Arranging of Things requires functions like Separate, Unite, gather and Spread.
The Data Scientists work with the data to separate a column like address which has state and city and put them into different columns. This is similar to separating shirts and pants. The Data scientist also would unite two similar columns together for better understanding such as Suffix and case number to fully view the claim number. This is similar to uniting shirts and t-shirts under same group.
Similarly, they also gather certain data together or spread the data into different columns based on the need and requirement. If the dresses are little far apart, we try to gather and place it in the right order. If some many dresses are piled up together, it is better to spread and sort in a more easier fashion.
These data tidying skills are required for data scientist to work on the data sets or tables and to logically present the data for better business decision making.
3. Select and Arrange things in order just like food grains processing to filter the good grains.
Similar to food grain processing, the data scientist work with data to select and arrange things in order. This is simply like filtering data until a particular year to see the sales process. Sometimes they need to see the weight in Kg to Pounds and may do extra mutations around the data. They also need to rename a field say temp to temperature for better understanding. These organizing skills when done technically in the data world, it would be necessary to reuse those basic raw human skills in the digital interface and that demands the job.
4. Corrective measures needs to be taken in our lives for correct Actions and better decision making
The Data correction is similar to our corrective measures we take in our lives such as closing the tap to save water or having a lock to the cupboard or a small sign such as wet area and caution. Similarly, when there is no data for a field say Last name, it is better to indicate a sign there such as NA or not available or NAN(Not an Number) in the data world. Then the use of probability and statistics would be applied to fill the NA with mean and median values.In this way, you are trying to better correct data and avoid confusions in those cases where the data is messy.
When we are completely not aware of a question being asked to us, we would simply say, "I don't know" . However in companies like IT or Data World, the data is required before hand to say something.Now we don't know the exact value but have a dirty data at least in our hand, we might say an average or rough value. These are the ways the decision making is applied in digital world.
Comments
Post a Comment