Notes
Notes - notes.io |
python
Copy code
import os
import subprocess
# Sourcing config.sh file if necessary
# . ../config.sh
# Assuming config.sh is sourced outside of this script
# Getting current working directory
cwd = os.getcwd()
# Setting the update file location
UPDATEFILELOCATION = "/home/donor/Documents/dbUpdateFiles/ConsumerFile"
# Change directory to UPDATEFILELOCATION
os.chdir(UPDATEFILELOCATION)
# List files in UPDATEFILELOCATION and save to tmp.csv
with open('tmp.csv', 'w') as f:
files = os.listdir()
for file in files:
f.write(file + 'n')
# Find the first file starting with "aa" and save its first line to headers.csv
first_file = next(file for file in os.listdir() if file.startswith("aa"))
with open(first_file, 'r') as f:
first_line = f.readline().strip()
with open('headers.csv', 'w') as f:
f.write(first_line + 'n')
# Change back to the original directory
os.chdir(cwd)
print("File rows loaded... performing migrations...")
# TODO: Perform migrations on the table based on added/removed columns
# TODO: Pull previous mapping, use that to determine columns to push to the new table
# TODO: Push data to the new table, set all to archived (preferably in the insert query)
# TODO: Threaded looping of the file, to perform insert/update actions
# TODO: Push all data to fresh consumer file mapping, preferably with migrations done already
# TODO: Start rolling update script on the fresh index
# TODO: Swap tables
# Run the MySQL migration script (Assuming the MySQL command is correct and accessible)
'''
subprocess.run([
'mysql',
'--local-infile=1',
'-h',
MYSQLLIVEHOST,
'-u',
MYSQLLIVEUSER,
'-p' + MYSQLLIVEPASS,
'donor_Logs',
'<',
'loadDataToDb.sql'
], shell=True)
'''
Explanation:
The Python script replicates the functionalities of your Bash script.
It handles file operations, directory navigation, and executing MySQL migration scripts.
The script uses the os module for directory and file operations, and subprocess module for executing shell commands.
The TODO comments indicate the points where you need to implement specific logic or functionalities as described in your original Bash script. These are placeholders for your custom Python logic.
///////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
Importing Modules:
python
Copy code
import os
import subprocess
These lines import necessary modules. os module provides a portable way of using operating system dependent functionality. subprocess module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes.
Setting Variables:
python
Copy code
cwd = os.getcwd()
UPDATEFILELOCATION = "/home/donor/Documents/dbUpdateFiles/ConsumerFile"
cwd stores the current working directory.
UPDATEFILELOCATION holds the path to the directory where update files are stored.
Change Directory and List Files:
python
Copy code
os.chdir(UPDATEFILELOCATION)
Changes the current working directory to UPDATEFILELOCATION.
List Files and Save to tmp.csv:
python
Copy code
with open('tmp.csv', 'w') as f:
files = os.listdir()
for file in files:
f.write(file + 'n')
Lists all files in the UPDATEFILELOCATION directory and writes their names to a file called tmp.csv.
Find First File and Save First Line to headers.csv:
python
Copy code
first_file = next(file for file in os.listdir() if file.startswith("aa"))
with open(first_file, 'r') as f:
first_line = f.readline().strip()
with open('headers.csv', 'w') as f:
f.write(first_line + 'n')
Finds the first file in the directory that starts with "aa".
Reads the first line of that file and saves it to a file called headers.csv.
Change Back to Original Directory:
python
Copy code
os.chdir(cwd)
Changes the working directory back to the original directory stored in cwd.
Print Information:
python
Copy code
print("File rows loaded... performing migrations...")
Prints a message indicating that file rows are loaded and migrations are about to be performed.
TODO Comments:
These comments highlight areas where you need to implement specific logic or functionalities as described in your original script. These serve as placeholders for your custom Python logic.
Run MySQL Migration Script (Commented Out):
The MySQL migration script is commented out and needs to be adapted based on your specific MySQL configuration and requirements.
Overall, the Python script replicates the functionality of the original Bash script, handling file operations, directory navigation, and executing MySQL migration scripts.
|
Notes is a web-based application for online taking notes. You can take your notes and share with others people. If you like taking long notes, notes.io is designed for you. To date, over 8,000,000,000+ notes created and continuing...
With notes.io;
- * You can take a note from anywhere and any device with internet connection.
- * You can share the notes in social platforms (YouTube, Facebook, Twitter, instagram etc.).
- * You can quickly share your contents without website, blog and e-mail.
- * You don't need to create any Account to share a note. As you wish you can use quick, easy and best shortened notes with sms, websites, e-mail, or messaging services (WhatsApp, iMessage, Telegram, Signal).
- * Notes.io has fabulous infrastructure design for a short link and allows you to share the note as an easy and understandable link.
Fast: Notes.io is built for speed and performance. You can take a notes quickly and browse your archive.
Easy: Notes.io doesn’t require installation. Just write and share note!
Short: Notes.io’s url just 8 character. You’ll get shorten link of your note when you want to share. (Ex: notes.io/q )
Free: Notes.io works for 14 years and has been free since the day it was started.
You immediately create your first note and start sharing with the ones you wish. If you want to contact us, you can use the following communication channels;
Email: [email protected]
Twitter: http://twitter.com/notesio
Instagram: http://instagram.com/notes.io
Facebook: http://facebook.com/notesio
Regards;
Notes.io Team