She Becomes Ugly if She Doesn’t Study

The question is very long, a whole page.

The front is a long introduction to the game.

Tang Su summed up that it is necessary to crawl, clean, organize, calculate, express and analyze the data of the virtual website provided by the competition question, and finally realize the data visualization in the form of charts.

Although it is only the preliminary round, Tang Su feels that the difficulty of the competition is really a bit difficult, especially since they are only freshmen, they have not learned some professional knowledge, and have not even taken courses related to data visualization.

Tang Su has taught himself some courses on data visualization, but he is not in-depth.

Tang Su gave the connection on the opening question, ready to start crawling data.

But before she could do it, she saw some classmates leave.

Tang Su took a look and saw that Yang Lu and Qiu Xiao, classmates in his dormitory, were among those who left. Nearly twenty or thirty classmates left, many of whom were their own.

Tang Su took a deep breath.

It seems that many students have no way to start this competition, or have to give up the competition because they have not mastered some relevant skills.

Tang Su didn't care about the others, she started to operate.

She first installed and deployed Hadoop-related components, mainly installing Hive components.

After the first step, she started scraping the data using the Python language.

Tang Su has also gone to some websites to crawl data before. This step is not very difficult for her. It is also one of the basic skills that a big data student needs to master.

In the second step, after crawling the data, Tang Su began to extract valid data, and then formatted the data into json format. Tang Su accomplished this step skillfully, because he had done it before.

The third step is to clean and analyze the data. This step is a very critical step. After thinking about it, Tang Su wrote a MapReduce program for data cleaning in java. After cleaning the data, she loads the available data into the Hive database, and completes data analysis and statistics by running HQL commands. Finally, execute the SQL script in Hive to view the data in the table.

This series of operations took a lot of time, and Tang Su saw that two hours had passed.

She only had one hour left to complete the challenge.

The fourth step is to complete the data visualization. After thinking about it, Tang Su uses histograms, line charts, and radar charts to output the data he analyzed.

The theme of this competition is to compare and analyze the salaries of employees in the IT industry in various places and draw the analysis results.

The fifth step is to write a data analysis report.

There is still half an hour before the end of the game.

At this point, less than one-third of the people still on the scene were left.

Many students either gave up the game and left directly, and some may have finished and left in advance.

With the visual chart, Tang Su's data analysis was relatively smooth, and he also successfully wrote the analysis report within the specified time.

After writing the report, Tang Su clicked submit and left the competition site.

The score will not be released on the spot, so Tang Su will have to wait for the list of the semi-finals announced in a few days to know whether he has a chance to advance to the semi-finals.

Nearly 150 people participated in the preliminary round, but only 30 students could enter the semi-finals. Tang Su didn't know if she had this chance, but she did her best and completed the questions according to the steps.

If she doesn't make the cut in the end, it can only be said that her current professional level is not enough.