Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File Size is increasing, when saving new mf4 file #1129

Open
gual11 opened this issue Jan 17, 2025 · 5 comments
Open

File Size is increasing, when saving new mf4 file #1129

gual11 opened this issue Jan 17, 2025 · 5 comments

Comments

@gual11
Copy link

gual11 commented Jan 17, 2025

We work with existing mf4 files, read out channels we need, modify the name and then write them into a new mf4 file. We work with compression= 1, mostly that works. Sometimes the file increases a little in size, which is ok. But sometimes file size is 5 to 10 times as huge as the original one. With a 500 MB file that results in some GB, which is a lot.

Can you tell, why this happens sometimes? Is it because of some signal format? actually because we get rid of some signals, we would expect the file size to be even lower than the original one.

@gual11
Copy link
Author

gual11 commented Jan 17, 2025

MDF Version 4.10
asammds Version 7.4.5

@danielhrisca
Copy link
Owner

Integer signals that are not a standard bit size will be saved as standard size ( 1 bit signal becomes 8 bit, 10 bit signal becomes 16 bit ).

Another increase is caused when VLSD signals are stored (variable length signal) because the biggest sample length will be used by the saved signal.

You should use compression=2 to get extra space saving

@gual11
Copy link
Author

gual11 commented Jan 17, 2025

thaks for the quick answer.
Even with comprassion = 2 we faced the same behaviour.
Regarding these VLSD signals, how do I identify them and can I compress them or influence their save sample size?

@danielhrisca
Copy link
Owner

Can you share how you create the new MDF ?

@gual11
Copy link
Author

gual11 commented Jan 17, 2025

mdf = asammdf.MDF()
for partnumber,subdict in data.items():
	sigs = []
	for channelname,channeldata in subdict.items():
		try:
			signal = asammdf.Signal(samples=channeldata[:,1],timestamps=channeldata[:,0]*10**(-9),name=channelname,comment=partnumber)
			sigs.append(signal)
		except Exception as e:
			print(channelname,e)
	mdf.append(sigs,acq_name=partnumber)
if status:
	mdf.header['abs_time'] = offset
	mdf.save(path2export,overwrite=True,compression=1)
mdf.close()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants