In this part of the assignment, you will write shell scripts to extract various types of texts from the Wikipedia arcitles downloaded in the first part. Use grep with regular expressions to complete the tasks. Your scripts should only output the matched texts instead of the full lines (hint: check man grep for options).
Task 0: Extracting Simple Links¶
A simple link is a span of text wrapped in double square brackets, [[...]], whose contents contain a page title with no pipe (|) or colon (:) (e.g. [[Weyerhaeuser]]).
Write a shell script extract_simple_links.sh that takes in a .wiki file and outputs all simple links.
$ ./extract_simple_links.sh Cosmo_Specialty_Fibers.wiki
[[Weyerhaeuser]]
[[The Gores Group]]
...Task 1: Extracting All Links¶
A link is a span of text wrapped in double square brackets, [[...]], whose contents contain a page title with no pipe (|) or colon (:), optionally followed by a pipe (|) and display text (e.g. [[title]] or [[title|display text]]).
Write a shell script extract_all_links.sh that takes in a .wiki file and outputs all links.
$ ./extract_all_links.sh Cosmo_Specialty_Fibers.wiki
[[Weyerhaeuser]]
[[The Gores Group]]
...
[[United States Environmental Protection Agency|Environmental Protection Agency]]
...Task 2: Extracting Categories¶
A category is a link whose contents begin with the prefix Category: (e.g. [[Category:Sawmills in the United States]]).
Write a shell script extract_categories.sh that takes in a .wiki file and outputs all categories.
$ ./extract_categories.sh Cosmo_Specialty_Fibers.wiki
[[Category:Pulp and paper mills in the United States]]
[[Category:Sawmills in the United States]]Task 3: Extracting Citations¶
A citation is a span of text wrapped in double curly braces, {{...}}, whose contents begin with Cite or cite (e.g. {{Cite web ...}}). For this assignment, we assume there are no nested double curly braces within a citation.
Write a shell script extract_citations.sh that takes in a .wiki file and outputs all citations.
$ ./extract_citations.sh Cosmo_Specialty_Fibers.wiki
{{Cite web |title=Red Harbor: The IWW in Grays Harbor, Washington - IWW History Project |url=https://depts.washington.edu/iww/red_harbor.shtml |access-date=April 13, 2026 |website=depts.washington.edu |archive-date=September 20, 2025 |archive-url=https://web.archive.org/web/20250920071127/https://depts.washington.edu/iww/red_harbor.shtml |url-status=live }}
{{Cite web |last=staff |first=Seattle Times |date=December 23, 2005 |title=83 lose jobs as Aberdeen sawmill closes |url=https://www.seattletimes.com/seattle-news/83-lose-jobs-as-aberdeen-sawmill-closes/ |access-date=April 10, 2026 |website=The Seattle Times |language=en-US |archive-date=April 21, 2026 |archive-url=https://web.archive.org/web/20260421094409/https://www.seattletimes.com/seattle-news/83-lose-jobs-as-aberdeen-sawmill-closes/ |url-status=live }}
...