Any Python Scripters here?

[Dennis Peacock]

I'm trying to learn Python on my own and I get the general gist of Python. I've been writting KSH and BASH scripts for a long time "and" I used to write COBOL many years ago. The issue I'm trying to figure out is I have a file that is 274,000 lines long. In "human" terms, the file is broken up into "records" logically speaking. A snippet is like this:



IMAGE blah 0 0 13 blah_1586185200 blah_U_FS 0 *NULL* root daily_onsite 1 3 1586185200 48 1589209200 0 0 2142496 41952 2 2 0 blah_U_FS_1586185200_INCR.f *NULL*
HISTO 0 0 0 0 0 0 0 0 0 0
FRAG 1 1 2142496 0 0 0 0 @zzzz app0x 262144 0 0 -1 1028 1;PureDisk;app0x;dp_disk_app0x;PureDiskVolume;0 1589209200 0 65539 0 0 0 6
FRAG 2 1 2142496 0 0 0 0 @zzzz cc0x 262144 0 0 -1 4 1;PureDisk_cc0x_ibmcoslan;dp_cc_cc0x;cloudcata;0 1589209200 0 65539 0 1 0 6 1589209200
IMAGE blah 0 0 13 blah_1586098800 blah_U_FS 0 *NULL* root daily_onsite 1 3 1586098800 44 1589122800 0 0 2121248 42040 2 2 0 blah_U_FS_1586098800_INCR.f *NULL*
HISTO 0 0 0 0 0 0 0 0 0 0
FRAG 1 1 2121248 0 0 0 0 @zzzz app0z 262144 0 0 -1 1028 1;PureDisk;app0z;dp_disk_app0z;PureDiskVolume;0 1589122800 0 65539 0 0 0 6 1589122800
FRAG 2 1 2121248 0 0 0 0 @zzzz cc0x 262144 0 0 -1 4 1;PureDisk_cc0z_ibmcoslan;dp_cc_cc0z;cloudcat;0 1589122800 0 65539 0 1 0 6 1589122800 1586100556
IMAGE blah 0 0 13 blah_1586012400 blah_U_FS 0 *NULL* root daily_onsite 1 3 1586012400 54 1589036400 0 0 2108288 42126 2 2 0 blah_U_FS_1586012400_INCR.f *NULL*
HISTO 0 0 0 0 0 0 0 0 0 0

I need to break each logical "record" from IMAGE to the end of all the HISTO/FRAG lines...up to the next "IMAGE". In my pee-brain...IMAGE is the start of the record and the next "IMAGE " is the beginning of the next record.
I need to do stuff with info provided inside each "record". I've searched and read a LOT about this online but I guess I'm simply not "getting it". Surely there has to be a simple code way to do this in Python!??!

[Bruce Lowekamp]

Strange place to look for python guidance :)

I wouldn't say I'm a python hacker, but using tell() and seek() is one way:


def ReadNextChunk(stream):
"""reads lines from stream until next line begins with IMAGE.
returns array of lines. Empty array at EOF"""
lineset = []
line = stream.readline()
if line == "":
return lineset
assert line.startswith("IMAGE")
lineset.append(line)
while True:
pos = stream.tell()
line = stream.readline()
if line == "":
return lineset
if line.startswith("IMAGE"):
stream.seek(pos)
return lineset
lineset.append(line)



file = open("chunks.txt", "rt", encoding="utf-8")
while True:
chunk = ReadNextChunk(file)
if len(chunk) == 0:
break
print ("next chunk is" + str(chunk))

[Bill McNiel]

And I thought this was going to be a thread about Monty and the Boys.

[Dennis Peacock]

Strange place to look for python guidance :)


Well, I know that SMC has a wide variety of professional career fields and expertise. I haven't had any real luck looking elsewhere with any reasonable simple solution. I don't like complicated code just for the sake of trying to prove ones ability to be complicated. Simple code is easier to troubleshoot and update by any that didn't write the original code in the first place. So I ask of the SMC folks because they are cool, knowledgable, and helpful....Much like you Bruce. :) So Thank you!

[Dennis Peacock]

Hey @Bruce Lowekamp,
I keep searching and finding this statement:

You will likely not find a lot of online forums that tolerate beginner questions in any language.

I've been cussed at for asking what they term as stupid questions. I've taught myself almost 100% of everything I know about anything. I'm motivated, I buy books and go through them but there are things that I need help with and I just don't get it from public forums. One guy told me that I needed to learn and understand ALL Python code modules! Now, tell me who "knows" _every_ Python code module?!
OK, I'm done being frustrated about this.....so it's back to searching and trying my own code.
Thank You very much Bruce for your willingness to help. I appreciate it very much.

[Monte Milanuk]

Dennis,

I don't have a specific answer for your particular need... but some place that you might check out that is more beginner-question friendly would be the /r/learnpython (https://www.reddit.com/r/learnpython/) and /r/learnprogramming (https://www.reddit.com/r/learnprogramming/) subs over on Reddit (https://www.reddit.com/).

Another resource that you may find valuable is 'Automate The Boring Stuff (http://automatetheboringstuff.com/)' (with Python). It's available both as a dead-tree book, ebook, or for free online at the site above, as well as via video course on Udemy. It doesn't get into things like classes and OOP, just what 'normal' people might need to get things done for day-to-day one-off scripts.