Problem
Write a bash script to calculate the frequency of each word in a text file words.txt.
For simplicity sake, you may assume:
- words.txt contains only lowercase characters and space ’ ’ characters.
- Each word must consist of lowercase characters only.
- Words are separated by one or more whitespace characters.
For example, assume that words.txt has the following content:
the day is sunny the the
the sunny is is
Your script should output the following, sorted by descending frequency:
the 4
is 3
sunny 2
day 1
Note:
Don’t worry about handling ties, it is guaranteed that each word’s frequency count is unique.
Solution
for n in `cat words.txt`;do echo $n;done | sort | uniq -c | sort -k 1 -r | awk '{print $2,$1}'
#the loop to echo words.txt word by word, then `sort` to gather same words, then use `uniq -c` to calculate words. And then using `sort -k 1 -r` to sort column 1 as key in a reverse way. Finally `awk` is for formatted output.
Other solutions
cat words.txt | tr -s ' ' '\n' | sort | uniq -c | sort -r | awk '{ print $2, $1 }'
#`tr -s` is to truncate the string, but only remaining one instance.
An error occurs saying that “syntax error with ‘done’.” It is due to some unsigned characters in the script. We can use cat -e filename
to check whether the script contains unexpected characters.
Reference
shell学习之逐行读取文本
http://blog.csdn.net/lliumt/article/details/21012069
在Bash中将字符串拆分成数组
https://blog.zengrong.net/post/1591.html
提取数据grep, cut, awk, sed, join, paste
https://chunqishi.wordpress.com/2010/02/24/%E8%BD%AC%EF%BC%9A%E6%8F%90%E5%8F%96%E6%95%B0%E6%8D%AEgrep-cut-awk-sed-join-paste/
使用AWK进行分割字符串以及截取字符串
http://blog.csdn.net/Amkio/article/details/40656991