Mastering Wildcards in shell script: Powerful Pattern Matching for Shell Scripts

Introduction to Wildcards in Bash

Wildcards, also known as glob patterns, are powerful tools in Bash scripting that allow you to work with multiple files or directories using pattern matching. They provide a concise way to specify sets of filenames with similar characteristics. In this article, we'll explore various wildcards and their applications in shell scripting.

Basic Wildcard Characters

The Asterisk (*)

The asterisk matches any number of characters (including zero).

# List all files in the current directory
ls *

# List all .txt files
ls *.txt

# Find all files starting with "log"
ls log*

The Question Mark (?)

The question mark matches exactly one character.

# List files with single-character extensions
ls *.?

# Match files like file1.txt, file2.txt, but not file10.txt
ls file?.txt

Square Brackets ([])

Square brackets match any single character within the brackets.

# Match files starting with a, b, or c
ls [abc]*

# Match files ending with numbers 0 through 9
ls *[0-9]

Advanced Wildcard Techniques

Negation with Square Brackets ([!])

Use [!] to match any character except those listed.

# Match files not starting with a, b, or c
ls [!abc]*

Character Classes

Use predefined character classes for common patterns.

# Match files with alphabetic characters
ls *[[:alpha:]]*

# Match files with numeric characters
ls *[[:digit:]]*

Extended Globbing

Enable extended globbing for more powerful patterns.

# Enable extended globbing
shopt -s extglob

# Match one or more occurrences of 'a'
ls +(a)*

# Match zero or one occurrence of 'b'
ls ?(b)*

# Match either 'file1' or 'file2'
ls @(file1|file2)

Wildcards in Shell Scripts

Wildcards are particularly useful in shell scripts for flexible file handling.

#!/bin/bash

# Process all .log files
for file in *.log; do
    if [ -f "$file" ]; then
        echo "Processing $file"
        # Add your processing logic here
    fi
done

# Backup all .txt and .csv files
for file in *.txt *.csv; do
    if [ -f "$file" ]; then
        cp "$file" "/backup/$(date +%Y%m%d)_$file"
        echo "Backed up $file"
    fi
done

Wildcards with Find Command

Combine wildcards with the 'find' command for powerful file searches.

# Find all .jpg files in subdirectories
find . -name "*.jpg"

# Find files modified in the last day
find . -name "*" -mtime -1

# Find and delete all .tmp files
find . -name "*.tmp" -delete

Practical Example: Log File Analyzer

Let's create a script that demonstrates the use of wildcards in a practical scenario:

#!/bin/bash

# Enable extended globbing
shopt -s extglob

# Function to process log files
process_logs() {
    local pattern="$1"
    local count=0
   
    for file in $pattern; do
        if [ -f "$file" ]; then
            echo "Analyzing $file"
            # Count error occurrences
            errors=$(grep -c "ERROR" "$file")
            echo "  Errors found: $errors"
            count=$((count + 1))
        fi
    done

    echo "Total files processed: $count"
}

# Process different types of log files
echo "Processing today's logs:"
process_logs "log_$(date +%Y%m%d)*.log"

echo -e "\nProcessing all Apache logs:"
process_logs "apache_+([0-9]).log"

echo -e "\nProcessing misc logs:"
process_logs "?(system|app)_log_*.log"

Best Practices and Considerations

• Always quote variables containing wildcards to prevent unexpected globbing.
• Use the nullglob option (shopt -s nullglob) to handle cases where no files match a pattern.
• Be cautious with wildcards in rm commands to avoid accidental deletions.
• Remember that wildcards do not match hidden files (starting with .) by default.

Conclusion

Mastering wildcards in Bash scripting allows you to create more flexible and powerful scripts. They are invaluable for handling files and directories efficiently, especially when dealing with large numbers of files or complex directory structures. By incorporating wildcards into your scripts, you can significantly enhance their functionality and adaptability.

In our next article, we'll explore advanced text processing techniques in Bash, including sed, awk, and regular expressions. Stay tuned for more insights into mastering shell scripting!