Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SplFileObject very high disk usage #17379

Open
Red-Owl opened this issue Jan 6, 2025 · 3 comments
Open

SplFileObject very high disk usage #17379

Red-Owl opened this issue Jan 6, 2025 · 3 comments

Comments

@Red-Owl
Copy link

Red-Owl commented Jan 6, 2025

Description

// use fseek

<?php
set_time_limit(0);
![disk](https://github.com/user-attachments/assets/f02f3400-3645-4a8f-9be4-b9bf3c6dfa49)



echo "loading";
$filepath = "Video Music.txt";

$file = new SplFileObject($filepath);
$n = 0;
$lined = Array();
$data1 = Array();
$datar = "";



$file_sql = $filepath.'.sql';
// The new person to add to the file

// Write the contents to the file, 
// using the FILE_APPEND flag to append the content to the end of the file
// and the LOCK_EX flag to prevent anyone else writing to the file at the same time

$link = mysqli_connect('127.0.0.1', 'wlab', 'xxxxxx', 'test', '3307');
if (!$link) {
    die('Not connected : ' . mysql_error());
}

// make foo the current db
/* $db_selected = mysqi_select_db('torrent', $link);
if (!$db_selected) {
    die ('Can\'t use foo : ' . mysql_error());
}
*/


while (false !== ($char = $file->fgetc())) {
    //echo "$char\n";
    $n++;
    $lined[] = $char;
    if($char == "\n"){
        $datar = implode('', $lined);
        $data1 = explode("|", $datar);
        if(count($data1)==2){
        $sql = "INSERT INTO magnet SET mn_exid=NULL,mn_name='".addslashes($data1[1])."',mn_size=NULL,mn_seed=NULL,mn_leker=NULL,mn_magnet='".trim($data1[0])."', mn_categ='".basename($filepath)."', mn_origin='BitSnoop', mn_date='2012-02-12 00:00:00';\r\n";
        // echo "\n<br />"
        // file_put_contents($file_sql, $sql, FILE_APPEND | LOCK_EX);
        mysqli_query($link,$sql);
    }else{
            echo $datar; 
        }
        $datar = ""; 
        $lined = Array();
    }
    /* if($n==5000){
        exit;
    } */
 
}


echo $n;

?>

### PHP Version

 8.3.12 

### Operating System

win 11
@cmb69
Copy link
Member

cmb69 commented Jan 6, 2025

How did you conclude that the disk usage is caused by SplFileObject::fgetc(), and not by the database server?

@Red-Owl
Copy link
Author

Red-Owl commented Jan 6, 2025

yes :)

@iluuu1994
Copy link
Member

@Red-Owl In that case, can you simplify the reproducer? The queries shouldn't be needed. The file is read in 8192 byte chunks, so this shouldn't be incredibly inefficient. Of course, if you read a huge file, the process will be IO bound and then high disk usage is expected.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants