Upload Windows Converter
This commit is contained in:
Michael Reber
parent
e0ceade4e0
commit
c95c1f38e0
64
README.md
64
README.md
@ -1,56 +1,40 @@
|
||||
# Convert videos in-place to h265 (HEVC)
|
||||
|
||||
This is a simple ruby script that will chuch through a directory converting all of the videos to use h.265 or HEVC in place. This works by converting the video file to \*.tmp.mp4, and then moving it to its original file name with the mp4 extension. It will delete the old version of the file.
|
||||
# FFmpeg-HEVC-Video-Converter 1.0
|
||||
|
||||
A PowerShell script to convert videos to the HEVC video format utilizing GPU hardware acceleration using FFmpeg for Windows.
|
||||
|
||||
## Usage
|
||||
The main benefit being that you can save disk space significantly (most of the time).
|
||||
|
||||
This is a long running ruby script, it makes calls to FFMPEG using a ruby gem to scrape metadata of videos, and transcode them. It works by appling some simple filters to create a list of videos that can be converted, and then works through that queue.
|
||||
## Space Saving Examples 💡
|
||||
|
||||
Change to the directory, where the config and script is located then run: `$ ruby convertVideos.rb start`
|
||||
- 2.5GB MP4 to 500MB HEVC MP4
|
||||
- 3GB MP4 to 800MB HEVC MP4
|
||||
|
||||
_Results vary and depend on the input video's format, bitrate etc._
|
||||
|
||||
## Example Config
|
||||
Please note the preceding colons are important. Also the file must be called config.yml
|
||||
## Minimum System Requirements
|
||||
|
||||
```
|
||||
:directory: /mnt/movies
|
||||
:min_age_days: 5
|
||||
:log_location: /home/user/logs/hevc_conversion.log
|
||||
:preset: slow
|
||||
:threads: 8
|
||||
:max_new_file_size_ratio: 0.9
|
||||
```
|
||||
- PC with at least 2 cores
|
||||
- Recent NVidia or AMD graphics card
|
||||
- Enough free disk space for resulting video files
|
||||
|
||||
| Configuration | Description |
|
||||
| -------- | -------- |
|
||||
| directory | The directory to recurse into. All files will be considered within that directory. |
|
||||
|min_age_days | How many days old does this file have to be to be considered for conversion. |
|
||||
|log_location| This Script is designed to run deetached in the background. As such the log location is the best way to figure out whatis going on and the status of the conversion |
|
||||
| preset | used to trade off between final file size, quality, and transcoding time. I recomend slow. See ffmpeg docs for more detail. |
|
||||
|threads | How many threads should be used for converting the file. |
|
||||
|max_new_file_size_ratio| Once transocing an individual file is finished, this script will make sure the output is smaller than the origional. Spesifically new file size <= Old file size * this value. Since transcoding always involves quality loss this value should be less than 1.0 |
|
||||
**VITAL: ALWAYS check the resulting video files, ending with "(HEVC)" for expected length and try them out in a video player checking for quality and smooth playback.**
|
||||
|
||||
## Encoding
|
||||
|
||||
## Coridnation
|
||||
When this script starts to convert a video, it creates a .filename.tmp.mp4 file that used the old files filename. This acts as kind of a lock because the exitsance of that file is checked before conversion. This also allows us to save state between runs without needing to share a database or other coridination servcie. That file is left behind if, for any reason, the conversion fails, the process is stopped, or if afterthe conversion the new HEVC file is not at least 10% smaller than the origional. The content is replace with an explination if possible.
|
||||
During encoding (conversion) it is normal for high **CPU** and **GPU** usage. Be sure to only run the script when the PC has no other processes hogging up resources.
|
||||
|
||||
It also makes it so multiple computers can run this script, provided they are all run against the same backing file system (SMB, NFS, etc.). There is a risk of duplicate work if two processes try to start to transcode the same file at the same time, but this risk is minimal for large libraries. To further mitigate, the list of candidate files is randomaized.
|
||||
The script encodes the files alongside the original with (HEVC) at the end of the filename.
|
||||
|
||||
## Decoding
|
||||
|
||||
## Setup
|
||||
The resulting HEVC videos require a more powerful PC to decode and playback, than the original would.
|
||||
|
||||
1. Instal Ruby 2.1.0+
|
||||
2. Install (ffmpeg)[https://ffmpeg.org/download.html] near version 3.4.4 `# apt-get install ffmpeg`
|
||||
3. `$ gem install bundler`
|
||||
4. `$ bundle install`
|
||||
5. Optional: Install screen or tmux. This is to allows it to run in the background after closing SSH on a server.
|
||||
6. Edit the script if you want.
|
||||
7. Run the script.
|
||||
8. Automate it e.g. with cron?
|
||||
## Script Usage
|
||||
|
||||
|
||||
## Disclaimers
|
||||
- Only use with videos you have the rights to copy
|
||||
- This will delete the original video, so use with care. Test with a test directory before running on your entire library.
|
||||
- Use a test file with all your media playing devices to ensure that they can handle HEVC encoding.
|
||||
1. Download FFmpeg for Windows: https://ffmpeg.zeranoe.com/builds/ (see screenshots below)
|
||||
2. Extract ffmpeg.exe to a known path/folder
|
||||
3. Download **convert_Videos.ps1** and create **video_file_list.txt** alongside it
|
||||
4. Make 4 edits in **convert_Videos.ps1** using PowerShell ISE
|
||||
5. Copy+paste full paths into **video_file_list.txt** and save
|
||||
6. Run **convert_Videos.ps1** to convert to HEVC
|
||||
|
||||
@ -1,6 +0,0 @@
|
||||
:min_age_days: 0
|
||||
:directory: /mnt/movies
|
||||
:log_location: ./log.txt
|
||||
:preset: slower
|
||||
:threads: 8
|
||||
:max_new_file_size_ratio: 0.9
|
||||
204
convertVideos.rb
204
convertVideos.rb
@ -1,204 +0,0 @@
|
||||
require 'rubygems'
|
||||
require 'streamio-ffmpeg'
|
||||
require 'fileutils'
|
||||
require 'logger'
|
||||
require 'yaml'
|
||||
require 'peach'
|
||||
|
||||
VID_FORMATS = %w[.avi .flv .mkv .mov .mp4]
|
||||
|
||||
# Loads config file:
|
||||
@config=YAML.load(File.read("./config.yml"))
|
||||
|
||||
# Logger setup stuff:
|
||||
@logger=Logger.new(@config[:log_location])
|
||||
@logger.level=Logger::INFO
|
||||
|
||||
@logger.info "\n New Run starting now......"
|
||||
@logger.info "Config is being used: #{@config}"
|
||||
|
||||
# Check the file age:
|
||||
def file_age(name)
|
||||
(Time.now - File.ctime(name))/(24*3600)
|
||||
end
|
||||
|
||||
|
||||
def seconds_to_s(total_time)
|
||||
total_time=total_time.to_int
|
||||
return [total_time / 3600, total_time/ 60 % 60, total_time % 60].map { |t| t.to_s.rjust(2,'0') }.join(':')
|
||||
end
|
||||
|
||||
# Creates a list (array) of all aged files:
|
||||
def get_aged_files(directory)
|
||||
out=[]
|
||||
Dir.foreach(directory){|file|
|
||||
next if file == '.' or file == '..' or file.start_with?('.')
|
||||
fileName=File.join(directory,file)
|
||||
if(File.file?(fileName)) then
|
||||
if VID_FORMATS.include? File.extname(file) then
|
||||
if(file_age(fileName)>=@config[:min_age_days]) then
|
||||
out<<fileName
|
||||
end
|
||||
end
|
||||
elsif File.directory?(fileName) then
|
||||
out+=get_aged_files(fileName)
|
||||
end
|
||||
}
|
||||
return out
|
||||
end
|
||||
|
||||
# Returns a hash:
|
||||
def get_candidate_files(possibileFiles)
|
||||
out={
|
||||
movies: [],
|
||||
runtime: 0
|
||||
}
|
||||
times=[]
|
||||
possibileFiles.peach(4) do |file|
|
||||
if does_video_need_conversion?(file)
|
||||
out[:movies]<<file
|
||||
movie=FFMPEG::Movie.new(file)
|
||||
times<<movie.duration
|
||||
end
|
||||
end
|
||||
times.each{|time|
|
||||
out[:runtime]=out[:runtime]+=time
|
||||
}
|
||||
out[:movies].shuffle!
|
||||
return out
|
||||
end
|
||||
|
||||
# Check if video needs conversion or not:
|
||||
def does_video_need_conversion?(file)
|
||||
movie=FFMPEG::Movie.new(file)
|
||||
if movie.valid? then
|
||||
if movie.video_codec!="hevc" then
|
||||
unless File.exist?(get_temp_filename(file))
|
||||
return true
|
||||
end
|
||||
end
|
||||
end
|
||||
return false
|
||||
end
|
||||
|
||||
def get_base_name(file)
|
||||
outFileName=File.join(
|
||||
File.dirname(file),
|
||||
"#{File.basename(file,'.*')}")
|
||||
end
|
||||
|
||||
def get_temp_filename(file)
|
||||
"#{File.join(
|
||||
File.dirname(file),
|
||||
".#{File.basename(file,'.*')}")}.tmp.mp4"
|
||||
end
|
||||
|
||||
def safe_convert_file(original_video,filename)
|
||||
begin
|
||||
return convert_file(original_video,filename)
|
||||
rescue StandardError => e
|
||||
@logger.error "Problem processing a video"
|
||||
@logger.error e
|
||||
end
|
||||
return nil
|
||||
end
|
||||
|
||||
# Main function for conversion:
|
||||
def convert_file(original_video,filename)
|
||||
options={
|
||||
video_codec: 'libx265',
|
||||
threads: @config[:threads],
|
||||
custom: "-preset #{@config[:preset]} -crf 25 -c:a copy".split
|
||||
}
|
||||
outFileName = get_base_name(filename)
|
||||
error_thrown=nil
|
||||
begin
|
||||
startTime=Time.now
|
||||
out = original_video.transcode(get_temp_filename(filename),options){ |progress|
|
||||
duration=Time.now-startTime
|
||||
remaining=(duration/progress)*(1-progress)
|
||||
if(remaining>99999999) then
|
||||
print "Progress converting #{filename.split('/').last} : #{(progress*100).round(1)}% \r"
|
||||
else
|
||||
print "Progress converting #{filename.split('/').last} : #{(progress*100).round(1)}% ETA is #{seconds_to_s(remaining)} \r"
|
||||
end
|
||||
}
|
||||
rescue StandardError => e
|
||||
error_thrown=e
|
||||
puts e.to_s
|
||||
end
|
||||
puts "Done with #{filename.split('\\').last}"
|
||||
if ( error_thrown )
|
||||
@logger.error "A video file failed to transcode correctly"
|
||||
@logger.error error_thrown
|
||||
FileUtils.rm(get_temp_filename(filename)) if File.exists?(get_temp_filename(filename))
|
||||
FileUtils.touch(get_temp_filename(filename))
|
||||
File.write(get_temp_filename(filename),
|
||||
[
|
||||
'An exception occured while transocding this movie.',
|
||||
error_thrown
|
||||
].join('\n'))
|
||||
elsif (out.size>original_video.size*@config[:max_new_file_size_ratio])
|
||||
@logger.warn "A video file, after transcoding was not at least #{@config[:max_new_file_size_ratio]} the size of the origional (new: #{out.size} old: #{original_video.size}). Keeping origonal #{filename}"
|
||||
FileUtils.rm(get_temp_filename(filename))
|
||||
FileUtils.touch(get_temp_filename(filename))
|
||||
File.write(get_temp_filename(filename), "transcoded video not enough smaller than the origional.")
|
||||
return nil
|
||||
else
|
||||
FileUtils.mv(get_temp_filename(filename),"#{outFileName}.mp4")
|
||||
if filename!="#{outFileName}.mp4" then
|
||||
FileUtils.rm(filename)
|
||||
end
|
||||
return out
|
||||
end
|
||||
end
|
||||
|
||||
def status(app)
|
||||
possible_files=get_aged_files(@config[:directory])
|
||||
puts "There are a total of #{possible_files.size} files that may need to be converted."
|
||||
|
||||
candidate_files= get_candidate_files(possible_files)
|
||||
|
||||
puts "There are a total of #{candidate_files[:movies].size} files that have not been converted yet."
|
||||
puts "Total Duration: #{seconds_to_s(candidate_files[:runtime])}"
|
||||
end
|
||||
|
||||
@total_processing_time=0
|
||||
@processed_video_duration=0
|
||||
|
||||
def iterate
|
||||
possible_files=get_aged_files(@config[:directory])
|
||||
@logger.info "There are a total of #{possible_files.size} files that may need to be converted."
|
||||
|
||||
@logger.debug "Files to be checked: #{possible_files}"
|
||||
|
||||
candidate_files= get_candidate_files(possible_files)
|
||||
|
||||
@logger.info "There are a total of #{candidate_files[:movies].size} files that have not been converted yet."
|
||||
@logger.debug "Candidate Files that need to be re-encoded: #{candidate_files}"
|
||||
@logger.info "Total Duration: #{seconds_to_s(candidate_files[:runtime])}"
|
||||
remaining_runtime=candidate_files[:runtime]
|
||||
|
||||
|
||||
candidate_files[:movies].each_with_index do |file,index|
|
||||
@logger.info "Starting to transcode file #{index+1} of #{candidate_files[:movies].size}: #{file}"
|
||||
unless does_video_need_conversion?(file)
|
||||
@logger.info "Video already converted, scanning again"
|
||||
return true
|
||||
end
|
||||
startTime=Time.now
|
||||
video=FFMPEG::Movie.new(file)
|
||||
converted_video=safe_convert_file(video,file)
|
||||
duration=Time.now - startTime
|
||||
remaining_runtime-=video.duration
|
||||
if !converted_video.nil? then
|
||||
@total_processing_time+=duration
|
||||
@processed_video_duration+=video.duration
|
||||
end
|
||||
avg=@processed_video_duration/@total_processing_time
|
||||
@logger.info "Average videotime/walltime: #{avg} Estimated time remaining #{seconds_to_s(remaining_runtime/avg)}"
|
||||
end
|
||||
return false
|
||||
end
|
||||
|
||||
while iterate do end
|
||||
98
convert_Videos.ps1
Normal file
98
convert_Videos.ps1
Normal file
@ -0,0 +1,98 @@
|
||||
# Converts videos to HEVC for all paths in "video_file_list.txt"
|
||||
|
||||
# Note: To copy paths in Windows File Explorer
|
||||
# Hold down the shift key and right click a selection of video files,
|
||||
# click "Copy as Path" and paste into "video_file_list.txt" and save.
|
||||
|
||||
# Ref: https://trac.ffmpeg.org/wiki/Encode/H.265
|
||||
|
||||
# Start Edits (4 in total)
|
||||
# Edit 1 of 4 - Set to the path of ffmepg.exe:
|
||||
$ffmpegEXE = 'Z:\Commands\ffmpeg.exe'
|
||||
# Edit 2 of 4 - Presets are ultrafast, superfast, veryfast,
|
||||
# faster, fast, medium, slow, slower, or veryslow:
|
||||
$preset = 'medium'
|
||||
# Edit 3 of 4: Video List File:
|
||||
$videosListFile = "$PSScriptRoot\video_file_list.txt"
|
||||
# Edit 4 of 4:
|
||||
# Default 'hevc_nvenc' is for recent nVidia GPU.
|
||||
# Set it to 'hevc_vaapi' if you are using a recent AMD video card instead.
|
||||
$hardwareEncoder = 'hevc_nvenc'
|
||||
# End Edits
|
||||
|
||||
# DO NOT EDIT BELOW HERE ==================================================
|
||||
|
||||
$arguments = ''
|
||||
|
||||
if(!(Test-Path $ffmpegEXE -PathType leaf))
|
||||
{
|
||||
Write-Host "ffmpeg.exe not found, please check path in `$ffmpegEXE." -ForegroundColor Yellow
|
||||
exit
|
||||
}
|
||||
|
||||
# Gets Video List File Contents
|
||||
$videos = Get-Content -Path $videosListFile
|
||||
$videoID = 1
|
||||
$count = $videos.Count
|
||||
if($videos.Count -lt 1)
|
||||
{
|
||||
Write-Host "No videos found in: $PSScriptRoot\video_file_list.txt" -ForegroundColor Red
|
||||
exit
|
||||
}
|
||||
|
||||
Write-Host "Converting $count videos.." -ForegroundColor Cyan
|
||||
|
||||
foreach($video in $videos)
|
||||
{
|
||||
# Converts using ffmpeg
|
||||
$video = $video.Replace("`"", "")
|
||||
$inputFile = $video
|
||||
#$inputFolder = (Get-Item $inputFile).Directory.FullName
|
||||
$outputFile = $video.Insert(($video.Length - 4), '(HEVC)')
|
||||
#$outputFolder = $inputFolder
|
||||
#$inputFile
|
||||
#$outputFile
|
||||
Write-Host
|
||||
Write-Host "Converting video ($videoID of $count), please wait.." -ForegroundColor Magenta
|
||||
Write-Host "$video `nto:`n$outputFile" -ForegroundColor White
|
||||
Write-Host
|
||||
|
||||
#$outputFilePath = "$outputFolder\$outputFileName" testing: -threads 2
|
||||
# hevc_nvenc
|
||||
# libx265
|
||||
$arguments = "-i `"$inputFile`" -hide_banner -y -xerror -threads 2 -c:a copy -c:s copy -c:v $hardwareEncoder " +
|
||||
"-crf 28 -preset $preset `"$outputFile`""
|
||||
#$arguments
|
||||
#$outputFile
|
||||
#exit
|
||||
#$videoID
|
||||
#Start-Process $ffmpegEXE -ArgumentList $arguments -WindowStyle Minimized -Wait
|
||||
Start-Process $ffmpegEXE -ArgumentList $arguments -WindowStyle Minimized
|
||||
$processName = 'ffmpeg'
|
||||
Start-Sleep -Seconds 8
|
||||
# Sets to use 3 cores (always set to one less core that your CPU has)
|
||||
# 2 Cores = 3, 3 Cores = 7, 4 cores = 15, 5 cores = 31, 6 cores = 63
|
||||
# Code to calculate for your CPU:
|
||||
# $noOfCores = Get-WmiObject Win32_Processor | Measure-Object NumberOfLogicalProcessors -Sum
|
||||
# $noOfCores.Sum = $noOfCores.Sum - 1
|
||||
# [math]::Pow(2,$($noOfCores).Sum) - 1
|
||||
#
|
||||
$process = Get-Process $processName; $process.ProcessorAffinity=7
|
||||
Start-Sleep -Seconds 8
|
||||
# Sets priorty to High
|
||||
# Values: High, AboveNormal, Normal, BelowNormal, Low
|
||||
$process = Get-Process -Id $process.Id
|
||||
$process.PriorityClass = 'High'
|
||||
|
||||
# Waits for process to complete
|
||||
$processID = (Get-Process $processName).id
|
||||
Wait-Process -Id $processID
|
||||
|
||||
# Increments video counter
|
||||
$videoID = $videoID + 1
|
||||
Start-Sleep -Seconds 4
|
||||
}
|
||||
|
||||
Write-Host "Finished converting $count videos." -ForegroundColor Green
|
||||
|
||||
exit
|
||||
Loading…
Reference in New Issue
Block a user